From fred.hebert@REDACTED Fri Jul 1 00:36:50 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 30 Jun 2011 18:36:50 -0400 Subject: [erlang-questions] json in erlang In-Reply-To: <4E0C1C56.6010708@gmail.com> References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> <4E0C1C56.6010708@gmail.com> Message-ID: <3E4C95EA-325D-4452-8162-4A8251FEACEC@erlang-solutions.com> I would definitely voice my support for jsx to become a standard if it were an option. -- Fred H?bert http://www.erlang-solutions.com On 2011-06-30, at 02:48 AM, Michael Truog wrote: > Is there an effort to help make jsx (https://github.com/talentdeficit/jsx) satisfy the "option lists" requirement, or is a different implementation being pursued? > > On 06/29/2011 10:11 PM, Richard O'Keefe wrote: >> On 30/06/2011, at 4:27 PM, Mihai Balea wrote: >>> There is at least one implementation of eep0018 in Github. >>> However people seem to prefer mochijson2. >> That's "partial implementation". It doesn't support the >> option lists described in EEP 18. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rzezeski@REDACTED Fri Jul 1 01:29:17 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 30 Jun 2011 19:29:17 -0400 Subject: [erlang-questions] GC memory consumption In-Reply-To: <4E040950.9000203@gmail.com> References: <4E040950.9000203@gmail.com> Message-ID: Dmitriy, If you know the amount you need ahead of time you could set the initial heap size at spawn and avoid GC altogether. -Ryan [Sent from my iPhone] On Jun 23, 2011, at 11:49 PM, Dmitriy Kargapolov wrote: > Hello, > > Erlang application we develop encountered memory lack problem (It's 32-bit unix system, so memory is limited by 2GB total per application). Analysing crash dumps I have found consistent scenario for the issue. Important facts: > > 1. Slogan: eheap_alloc: Cannot allocate 298930300 bytes of memory (of type "heap"). > > 2. All the processes are in 'waiting' state except one which is in 'Garbing' state. > > 3. The 'Garbing' process 'Stack+heap' is 59786060 bytes, OldHeap is 0 bytes, some fragment data available (174663 bytes or so). > > 4. System has also pretty large mnesia tables, this explains why there is not enough memory to allocate 300 MB segment at some point of execution. > > Looking at the GC code I found that it always allocates new heap segment to do its work. So in our consistent pattern GC tries to allocate 298930300 bytes to perform garbage collection for the process which heap is 59786060 bytes. > > Looks ridiculous - to de-fragment ~60 MB of the process heap, GC needs additional ~300 MB segment... This makes very difficult to design processes which have to keep large amount of data it its space (for example big tree-like structure or in-memory graph). > > Is there any known workaround for this (except moving it to 64 bit OS) ? > > Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From emmiller@REDACTED Fri Jul 1 02:17:56 2011 From: emmiller@REDACTED (Evan Miller) Date: Thu, 30 Jun 2011 19:17:56 -0500 Subject: [erlang-questions] How to catch a killer? Message-ID: I have gen_server that is being killed predictably but mysteriously (its supervisor receives a "killed" message). Are there any debugging tools available for figuring out who is sending the exit(kill) message? Tracing the process reveals nothing. The bug is probably something dumb in my code, but I'm surprised that a process can assassinate another process and leave behind no evidence. How can I catch the killer? From watson.timothy@REDACTED Fri Jul 1 02:44:08 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 1 Jul 2011 01:44:08 +0100 Subject: [erlang-questions] json in erlang In-Reply-To: <3E4C95EA-325D-4452-8162-4A8251FEACEC@erlang-solutions.com> References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> <4E0C1C56.6010708@gmail.com> <3E4C95EA-325D-4452-8162-4A8251FEACEC@erlang-solutions.com> Message-ID: 2011/6/30 Fr?d?ric Trottier-H?bert : > I would definitely voice my support for jsx to become a standard if it were an option. +1. I'm all for NIF based stuff when the speed is necessary, but I have a suspicion it'll take longer to get a native implementation into Erlang/OTP-proper and a pure library that is "fast enough" might be a better/faster option in terms of having something that is "there" when it's needed. If OTP was going to come with a native code implementation as a BIF, that would obviously be dead cool though. From michal.ptaszek@REDACTED Fri Jul 1 08:50:09 2011 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Fri, 1 Jul 2011 08:50:09 +0200 Subject: [erlang-questions] How to catch a killer? In-Reply-To: References: Message-ID: Hi Evan, try to play with dbg: dbg:tracer(), dbg:p(all, call), dbg:tp(erlang, exit, 2, []). If you know the pid of the gen_server that is killed, you might exchange '[]' with dbg:fun2ms and provide a pattern that will match only on that process: Pid = pid(X, Y, Z), %% Pid of your gen_server dbg:tp(erlang, exit, 2, dbg:fun2ms(fun([Pid, kill]) -> ok end)). Best regards, Michal Ptaszek On Jul 1, 2011, at 2:17 AM, Evan Miller wrote: > I have gen_server that is being killed predictably but mysteriously > (its supervisor receives a "killed" message). Are there any debugging > tools available for figuring out who is sending the exit(kill) > message? Tracing the process reveals nothing. > > The bug is probably something dumb in my code, but I'm surprised that > a process can assassinate another process and leave behind no > evidence. How can I catch the killer? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eeajam@REDACTED Fri Jul 1 09:08:35 2011 From: eeajam@REDACTED (ajam) Date: Fri, 1 Jul 2011 00:08:35 -0700 (PDT) Subject: [erlang-questions] erlang downloads Message-ID: <9fc4c746-fdb6-4012-86c0-d9bcc5a1c0eb@28g2000yqu.googlegroups.com> Just wonder if anyone has had problems lately downloading "erlang" from it's download site. Thanks! From carlo.bertoldi@REDACTED Fri Jul 1 11:10:03 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Fri, 01 Jul 2011 11:10:03 +0200 Subject: [erlang-questions] Url decoding Message-ID: <4E0D8EEB.2070902@ubiquity.it> Hello again, I believe this is a quite common problem, but I couldn't find an answer to my problem. I'd like to URL encode/decode strings. Encoding: edoc_lib:escape_uri("hello hello"). Perfect, but how can I go back to the original string now? Is it possible to do it without using external libs, as those found in Yaws, or ibrowse? or I'll have to include one of those in my project? Cheers, Carlo From essen@REDACTED Fri Jul 1 12:50:43 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 01 Jul 2011 12:50:43 +0200 Subject: [erlang-questions] Url decoding In-Reply-To: <4E0D8EEB.2070902@ubiquity.it> References: <4E0D8EEB.2070902@ubiquity.it> Message-ID: <4E0DA683.6020309@dev-extend.eu> On 07/01/2011 11:10 AM, Carlo Bertoldi wrote: > Hello again, > I believe this is a quite common problem, but I couldn't find an > answer to my problem. > I'd like to URL encode/decode strings. > Encoding: edoc_lib:escape_uri("hello hello"). Perfect, but how can I go > back to the original string now? > Is it possible to do it without using external libs, as those found in > Yaws, or ibrowse? or I'll have to include one of those in my project? You could use ex_uri as a rebar dependency if you're using rebar: https://github.com/extend/ex_uri -- Lo?c Hoguin Dev:Extend From chandrashekhar.mullaparthi@REDACTED Fri Jul 1 14:17:38 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 1 Jul 2011 13:17:38 +0100 Subject: [erlang-questions] Url decoding In-Reply-To: <4E0D8EEB.2070902@ubiquity.it> References: <4E0D8EEB.2070902@ubiquity.it> Message-ID: On 1 July 2011 10:10, Carlo Bertoldi wrote: > Hello again, > I believe this is a quite common problem, but I couldn't find an answer to > my problem. > I'd like to URL encode/decode strings. > Encoding: edoc_lib:escape_uri("hello hello"). Perfect, but how can I go > back to the original string now? > Is it possible to do it without using external libs, as those found in > Yaws, or ibrowse? or I'll have to include one of those in my project? > > The http_uri module which is part of inets is probably what you need. cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Fri Jul 1 14:23:13 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Fri, 01 Jul 2011 14:23:13 +0200 Subject: [erlang-questions] Url decoding In-Reply-To: References: <4E0D8EEB.2070902@ubiquity.it> Message-ID: <4E0DBC31.6000504@ubiquity.it> On 01/07/2011 14:17, Chandru wrote: > On 1 July 2011 10:10, Carlo Bertoldi > wrote: > > Hello again, > I believe this is a quite common problem, but I couldn't find an > answer to my problem. > I'd like to URL encode/decode strings. > Encoding: edoc_lib:escape_uri("hello hello"). Perfect, but how can > I go back to the original string now? > Is it possible to do it without using external libs, as those > found in Yaws, or ibrowse? or I'll have to include one of those in > my project? > > > The http_uri module which is part of inets is probably what you need. > > cheers > Chandru > Hi Chandru, I looked into http_uri module, but it only exports the parse function: http_uri:parse("http://try.com/ciao%20ciao"). {http,[],"prova.com",80,"/ciao%20ciao",[]} This doesn't solve my problem. From chandrashekhar.mullaparthi@REDACTED Fri Jul 1 14:32:57 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 1 Jul 2011 13:32:57 +0100 Subject: [erlang-questions] Url decoding In-Reply-To: <4E0DBC31.6000504@ubiquity.it> References: <4E0D8EEB.2070902@ubiquity.it> <4E0DBC31.6000504@ubiquity.it> Message-ID: On 1 July 2011 13:23, Carlo Bertoldi wrote: > On 01/07/2011 14:17, Chandru wrote: > > On 1 July 2011 10:10, Carlo Bertoldi > carlo.bertoldi@**ubiquity.it >> wrote: >> >> Hello again, >> I believe this is a quite common problem, but I couldn't find an >> answer to my problem. >> I'd like to URL encode/decode strings. >> Encoding: edoc_lib:escape_uri("hello hello"). Perfect, but how can >> I go back to the original string now? >> Is it possible to do it without using external libs, as those >> found in Yaws, or ibrowse? or I'll have to include one of those in >> my project? >> >> >> The http_uri module which is part of inets is probably what you need. >> >> cheers >> Chandru >> >> Hi Chandru, I looked into http_uri module, but it only exports the parse > function: > > http_uri:parse("http://try.**com/ciao%20ciao > "). > {http,[],"prova.com",80,"/**ciao%20ciao",[]} > > This doesn't solve my problem. > > I thought you wanted to URL encode and decode strings. So are the encode and decode functions in that module not sufficient? 8> edoc_lib:escape_uri("hello hello"). "hello%20hello" 9> http_uri:decode(v(-1)). "hello hello" Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Fri Jul 1 14:39:29 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Fri, 01 Jul 2011 14:39:29 +0200 Subject: [erlang-questions] Url decoding In-Reply-To: References: <4E0D8EEB.2070902@ubiquity.it> <4E0DBC31.6000504@ubiquity.it> Message-ID: <4E0DC001.6060704@ubiquity.it> On 01/07/2011 14:32, Chandru wrote: > > I thought you wanted to URL encode and decode strings. So are the > encode and decode functions in that module not sufficient? > > 8> edoc_lib:escape_uri("hello hello"). > "hello%20hello" > > 9> http_uri:decode(v(-1)). > "hello hello" > > Chandru > > That's exactly what I'm looking for. I got it, I'm using R14A, but that functions has been introduced in later versions. Many thanks, Carlo From chandrashekhar.mullaparthi@REDACTED Fri Jul 1 15:44:20 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 1 Jul 2011 14:44:20 +0100 Subject: [erlang-questions] mnesia lookup weirdness Message-ID: Hi, One of my production nodes running R12B-5 is exhibiting some strange behaviour. mnesia:dirty_read(gtp_tid, mnesia:dirty_first(gtp_tid)). gives me an empty list. The table has 39K entries. How do I find out what is happening? I tried using ets operations directly and get the same result (which is what mnesia is doing behind the scenes anyway). I get the same empty result using a transaction to read. Any clues? 96> ets:info(gtp_tid). [{memory,213857900}, {owner,<0.741.0>}, {name,gtp_tid}, {size,39088}, {node,node@REDACTED}, {named_table,true}, {type,set}, {keypos,2}, {protection,public}] The same operation works ok on other tables. cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jul 1 16:32:15 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 01 Jul 2011 16:32:15 +0200 Subject: [erlang-questions] Url decoding In-Reply-To: References: <4E0D8EEB.2070902@ubiquity.it> <4E0DBC31.6000504@ubiquity.it> Message-ID: <4E0DDA6F.8020004@dev-extend.eu> > I thought you wanted to URL encode and decode strings. So are the encode > and decode functions in that module not sufficient? > > 8> edoc_lib:escape_uri("hello hello"). > "hello%20hello" > > 9> http_uri:decode(v(-1)). > "hello hello" Note that while those functions are there, they are AFAIK not publicly documented and could change or be removed at any time in the future. -- Lo?c Hoguin Dev:Extend From dgriff1@REDACTED Sat Jul 2 23:14:45 2011 From: dgriff1@REDACTED (Daniel Griffin) Date: Sat, 2 Jul 2011 16:14:45 -0500 Subject: [erlang-questions] SSL {error, closed} Message-ID: I am using SSL sockets to talk between an erlang program and a twisted python program. Different processes in my erlang app occasionally have to open connections and the first one opens fine, then the second errors with {error, closed}. I don't know if I am misunderstanding something in SSL or Erlang. The simplest client code I could come up with is: run_test()-> ssl:start(), %%% works fine {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, {keepalive, true}, {active, false}, { certfile, "controller.crt"}, {keyfile, "controller.key"} ]), %%% this next one will return {error, closed} {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, 0}, {keepalive, true}, {active, false}, { certfile, "controller.crt"}, {keyfile, "controller.key"} ]). The python side gets the connection both times. Bizarrely, if I retry a connection after getting {error, closed} it works. Any insight? Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sun Jul 3 02:38:59 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sat, 2 Jul 2011 17:38:59 -0700 Subject: [erlang-questions] Scoped group leaders Message-ID: Hi, I had an idea to play with this long weekend and I want to share some results and get some feedback. Basically, I was wondering whether it will be feasible to extend the concept of group leaders beyond I/O so that you can build your own primitives around this construct of inherited group leaders. It seemed to me that tasks like implicit configuration, security groups and such can benefit from having a more generalized group leadership mechanism. In the last two days I wrote a quick-n-dirty proof of concept implementation (https://github.com/spawngrid/otp/tree/group_leader_scope), it's pretty rough but seems to work. I also wrote an early draft of an EEP for this feature (https://github.com/spawngrid/eep/blob/scoped-group-leaders/eeps/eep-scoped-group-leaders.md). The EEP has some examples, and you can play with that PoC implementation (group_leader_scope branch). It's not ready for the prime time, though ? it lacks automated tests and the performance implications have not yet been measured (although this should be fixed soon). No optimizations were considered at this moment. The EEP draft itself has not been sent to EEP editors as I plan to elaborate on motivations and possibly add more examples. Either way, please let me know if this is of any interest to anybody and if you have any thoughts about it. Have a nice weekend, Yurii. From peralta.alejandro@REDACTED Sun Jul 3 05:55:15 2011 From: peralta.alejandro@REDACTED (Ale) Date: Sun, 3 Jul 2011 00:55:15 -0300 Subject: [erlang-questions] How to write a generic IRC client the OTP way? In-Reply-To: References: Message-ID: Hello Jon, Thanks for your answer, sorry for the delay. > Also, for synchronous calls (where you want the status), waiting the > necessary amount of time is pretty much what you expect to do. You *can* > make arrangements to use cast, and pass in a respond-to-PID parameter as > part of the request, but in most cases, that adds significant extra > complexity for very little benefit. I think one of the problems I'm having is that I'm confusing synchronous calls of erlang processes and the actual process of the interpreter. I was wondering if I make a synchronous call from a wx process to a gen_server process (using gen_server:call for example) if the whole wx application would block. For example, would I be able to click a button mean while the call completes? I suppose that depends on the application, wouldn't it? I mean if I spawn another process to call gen_server the wx wouldn't block. > > Note that gen_server:call also takes a timeout parameter -- it defaults to > 5,000 milliseconds, so if the gen_server is "wedged" then the timeout will > kick in. > I'll see into it. Thanks again for your answer, -- Ale. From peralta.alejandro@REDACTED Sun Jul 3 06:02:21 2011 From: peralta.alejandro@REDACTED (Ale) Date: Sun, 3 Jul 2011 01:02:21 -0300 Subject: [erlang-questions] How to write a generic IRC client the OTP way? In-Reply-To: References: Message-ID: Hello Mazen, 2011/6/30 Mazen Harake : > Try eirc here: https://github.com/mazenharake/eirc > > Trying to do synchronous IRC is doomed to fail imho. eirc sends a > message everytime you get an event/response and by keeping your own > state you put context to these events to know what they mean. Thanks, I'll be checking it out, sounds very intersting. > IRC wasn't really made for computer programs but rather for a telnet > prompt so that is why this protocol is flawed in many ways (but very > easy). > It's also fun :-) > eirc is under the 2-clause BSD license so do what ever you want with it. That's great! Thanks for your help, -- Ale. From roberto@REDACTED Sun Jul 3 10:10:16 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 3 Jul 2011 10:10:16 +0200 Subject: [erlang-questions] spec declaration for single list parameter with fixed size Message-ID: dear list, a very stupid spec declaration question. how can you declare the specs of a function which has a single list with multiple arguments? For instance, consider this function: myfun([One, Two]) -> these two following declarations do obviously not work: -spec myfun([One::string(), Two::integer()]) -> -spec myfun(list(One::string(), Two::integer())) -> the proper way seems to be: -spec myfun([string() | integer()]) -> however i feel this to be very misleading, as the specs do not show the order nor the fixed length of the parameter list. is there a better way to do so? thank you. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttmrichter@REDACTED Sun Jul 3 11:58:34 2011 From: ttmrichter@REDACTED (Michael Richter) Date: Sun, 3 Jul 2011 17:58:34 +0800 Subject: [erlang-questions] spec declaration for single list parameter with fixed size In-Reply-To: References: Message-ID: On 3 July 2011 16:10, Roberto Ostinelli wrote: > a very stupid spec declaration question. how can you declare the specs of a > function which has a single list with multiple arguments? > For instance, consider this function: > > myfun([One, Two]) -> > > these two following declarations do obviously not work: > -spec myfun([One::string(), Two::integer()]) -> > -spec myfun(list(One::string(), Two::integer())) -> > > the proper way seems to be: > -spec myfun([string() | integer()]) -> > > however i feel this to be very misleading, as the specs do not show the > order nor the fixed length of the parameter list. > If you have a fixed length and format, it strikes me that a list isn't the data structure you want given that a list is by definition variable-length. Is there a reason you're not using a tuple instead of that list? Something like this?: myfun({One, Two}) -> ... -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 3 12:20:15 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 3 Jul 2011 12:20:15 +0200 Subject: [erlang-questions] spec declaration for single list parameter with fixed size In-Reply-To: References: Message-ID: On Sun, Jul 3, 2011 at 11:58, Michael Richter wrote: > On 3 July 2011 16:10, Roberto Ostinelli wrote: >> >> a very stupid spec declaration question. how can you declare the specs of >> a function which has a single list with multiple arguments? >> >> For instance, consider this function: > > >> >> myfun([One, Two]) -> > This is better since you then have a product type rather than a recursive type on lists. It is much simpler to work with from a typing perspective. Also note that you would not easily be able to type ["hello", 5] in a statically typed language without resorting to either some kind of type tagging, polymorphic variants or existential types. My advice would be to alter the structure of the function such that it is easier to type. It tends to be safer from a programming perspective as well. -- J. From brum76@REDACTED Sun Jul 3 18:45:25 2011 From: brum76@REDACTED (Radu Brumariu) Date: Sun, 3 Jul 2011 09:45:25 -0700 Subject: [erlang-questions] Spawnfest Erlang Contest reminder! In-Reply-To: <92323D9F-A131-48A5-9FD7-DCCEAB57AB17@erlang-solutions.com> References: <92323D9F-A131-48A5-9FD7-DCCEAB57AB17@erlang-solutions.com> Message-ID: Hi, I checked the website, but I couldn't find anything related to the criteria used in judging. Can you point me to where that is explained ? Thanks, Radu 2011/6/23 Fr?d?ric Trottier-H?bert : > Hey there everyone, > > This is a reminder for all interested people that spawnfest is going to take place in two weeks from today. Spawnfest is the first edition of an annual 48 hour Erlang development competition where you can compete against other developers in building the best, most breathtaking Erlang application, library or anything you like. > > The contest aims to let people flex their Erlang muscles and showcase awesome things they can do, but also aims to bring the community together and boost the interest towards the language. Teams of one to four members can subscribe to the contest by submitting the required info to our form (http://spawnfest.com/take-part/). Note that if you don't have a team and do not want to be a team of one, we have another form to help find potential teammates (http://spawnfest.com/find-a-team/). > > The contest commences at midnight Saturday 9th of July, 2011 00:00:00 GMT+0 and ends on Sunday 10th of July, 2011 23:59:59 GMT+0. The full set of rules is available at http://spawnfest.com/rules/ > > Our judging team is: > - Bob Ippolito, CTO and co-founder of Mochi Media Inc; > - Justin Sheehy, CTO at Basho Technologies; > - Robert Virding, Co-Inventor of Erlang and Erlang Solutions Principal Language Expert; > - Ulf Wiger, CTO of Erlang Solutions. > > And we have prizes! See them on our prize page (http://spawnfest.com/prizes/). We're always hunting for more prizes, so let us know if you're interested in providing any. > > I'll leave you with this little 'promotional' video: http://vimeo.com/22360540, and I hope we've sparked your interest. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From rvg.mailing@REDACTED Sun Jul 3 21:24:56 2011 From: rvg.mailing@REDACTED (Rudolph van Graan) Date: Sun, 03 Jul 2011 20:24:56 +0100 Subject: [erlang-questions] Getting eunit to display the full term? Message-ID: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> Hi there, Is there a simple way (other than changing the source) to make eUnit display the entire term that didn't match in its output instead of the ... truncation? I find it pointless to have write a manual version of the test just to see what is not matching. Thanks, Rudolph Compiled src/validator.erl validator:70: duplicate_check_test_...*failed* ::error:{assertMatch_failed, [{module,validator}, {line,71}, {expression, "parser : compile_and_validate ( message ( 2 ) )"}, {expected,"ok"}, {value, {error, [{specification_error,duplicate_element, "Duplicate", [{element_spec,12,...}, {element_spec,...}]}, {specification_error,duplicate_element, "Elements", [{element_spec,...},{...}]}]}}]} in function validator:'-duplicate_check_test_/0-fun-2-'/0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Jul 3 22:17:03 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 3 Jul 2011 21:17:03 +0100 Subject: [erlang-questions] Getting eunit to display the full term? In-Reply-To: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> References: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> Message-ID: On 3 July 2011 20:24, Rudolph van Graan wrote: > Hi there, > > Is there a simple way (other than changing the source) to make eUnit display > the entire term that didn't match in its output instead of the ... > truncation? I find it pointless to have write a manual version of the test > just to see what is not matching. > You could use a hamcrest matcher from within the eunit test case. That would at return a formatted (an not truncated) representation. You can also set up a "heckle" function to notify what is being checked and against which expectations. The library is quite early stages, but I'll happily fix issues and/or take pull requests to make it more useful. t4@REDACTED:~ $ erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> X = [{specification_error,duplicate_element,"Duplicate", 1> [{element_spec,12,[{foo,bar},baz]}, 1> {element_spec,14,[{a,b},{c,d}]}]}] 2> Result = hamcrest:check([X,X,X,X], hamcrest_matchers:equal_to(abc)). {assertion_failed,"Expected a value equal to [abc], but was [[[{specification_error,\n duplicate_element,\"Duplicate\",\n [{element_spec,12,\n [{foo,bar},baz]},\n {element_spec,14,\n [{a,b},{c,d}]}]}],\n [{specification_error,\n duplicate_element,\"Duplicate\",\n [{element_spec,12,\n [{foo,bar},baz]},\n {element_spec,14,\n [{a,b},{c,d}]}]}],\n [{specification_error,\n duplicate_element,\"Duplicate\",\n [{element_spec,12,\n [{foo,bar},baz]},\n {element_spec,14,\n [{a,b},{c,d}]}]}],\n [{specification_error,\n duplicate_element,\"Duplicate\",\n [{element_spec,12,\n [{foo,bar},baz]},\n {element_spec,14,\n [{a,b},{c,d}]}]}]]]"} 3> From g@REDACTED Mon Jul 4 07:01:12 2011 From: g@REDACTED (Garrett Smith) Date: Sun, 3 Jul 2011 23:01:12 -0600 Subject: [erlang-questions] Getting eunit to display the full term? In-Reply-To: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> References: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> Message-ID: On Sun, Jul 3, 2011 at 1:24 PM, Rudolph van Graan wrote: > Hi there, > > Is there a simple way (other than changing the source) to make eUnit display > the entire term that didn't match in its output instead of the ... > truncation? I find it pointless to have write a manual version of the test > just to see what is not matching. It's not elegant, but I insert io:format("~p", [Result]) in places where the result gets truncated. Output via io:format is suppressed when the test succeeds, suggesting that this is somehow a feature. I would argue though that this is a design flaw in eunit -- results should not be truncated. Garrett From pablo.platt@REDACTED Mon Jul 4 08:05:55 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 3 Jul 2011 23:05:55 -0700 (PDT) Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case Message-ID: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> I read about the pros and cons of pmods and wanted to find an alternative. In mochiweb, pmods doesn't look essential. You can replace all the Req:something type of calls with mochiweb_request:something(Req, Args) Chicago Boss uses pmods in the following way: (http://www.evanmiller.org/chicago-boss-guide.html): >Greetings = boss_db:find(greeting, []). [{greeting, "greeting-1", "Boss says Hello!"}] >Greeting = hd(Greetings). {greeting, "greeting-1", "Boss says Hello!"} >Greeting:greeting_text(). "Boss says Hello!" The powerful part is being able to use the model in an erl_dtl template: {% for greeting in greetings %}
  • {{ greeting.text }}
    • {% endfor %} Using pmods you can pass different types of models with common fields and use them in the same template. For example, models of several product types that have name and price fields. You can also load data dynamically in the template. A many-to-many field(function) in the pmod can load data from the db and pass an 'instance' of another pmod. Alternative to pmods in that case could be records, proplist/dict, gen_server. - records To use a record, we need to tell the template about the record type. {% for p#product in products %} That's fine but we can only use the same template for several model types if the records of all product types have the same structure: {product_type1, {name, price, some_field}} {product_type2, {name, price, another_field}} We can't 'extend' a record in erlang so we'll have to manually make write the structure of all related records which will be hard in a large application. - proplist/dict This will probably work but I'm not sure about performance. For small models' we probably want to use proplists and for large models a dict. I'm not sure what small and large are in this case. We can't dynamically load data in the template unless we define the model type in the template. {% for product#some_type in group_product.children %} - gen_server Might work but creating 100 gen_servers just to show 100 products feels like big overhead. Any ideas? Thanks From watson.timothy@REDACTED Mon Jul 4 09:53:34 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 4 Jul 2011 08:53:34 +0100 Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> Message-ID: > Alternative to pmods in that case could be records, proplist/dict, gen_server. > > - records > > To use a record, we need to tell the template about the record type. > {% for p#product in products %} > That's fine but we can only use the same template for several model types > if the records of all product types have the same structure: > {product_type1, {name, price, some_field}} > {product_type2, {name, price, another_field}} > We can't 'extend' a record in erlang so we'll have to manually make write the structure of all related records which will be hard in a large application. > You could extend (or rewrite) something like https://github.com/esl/parse_trans/blob/master/doc/exprecs.md to export a bunch of functions associated with the record. This stuff works at compile time (using parse transforms) but could be achieved at runtime with a bit of beam munging. From magnus.henoch@REDACTED Mon Jul 4 10:50:39 2011 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Mon, 4 Jul 2011 08:50:39 +0000 (GMT) Subject: [erlang-questions] Getting eunit to display the full term? In-Reply-To: <851607877.48081309769404158.JavaMail.root@zimbra> Message-ID: <619661083.48101309769439529.JavaMail.root@zimbra> ----- "Rudolph van Graan" wrote: > Hi there, > > Is there a simple way (other than changing the source) to make eUnit > display the entire term that didn't match in its output instead of the > ... truncation? I find it pointless to have write a manual version of > the test just to see what is not matching. I usually look at the report generated by eunit_surefire. It used to be limited in the same way as the terminal output, but I submitted a patch to increase the depth to 100 (used to be 20). It was included in R14B03. https://github.com/erlang/otp/commit/b6decafc7a45e7519dd53d1956909463a06d3f6a Magnus From ingela.andin@REDACTED Mon Jul 4 11:00:23 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 4 Jul 2011 11:00:23 +0200 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: Hi! It sounds like you are experiencing some kind of timing problem. I tried to write a test case to repeat it but I am not getting any problem. Which ssl-version you are running? Regards Ingela Erlang/OTP team - Ericsson AB 2011/7/2 Daniel Griffin : > I am using SSL sockets to talk between an erlang program and a twisted > python program. Different processes in my erlang app occasionally have to > open connections and the first one opens fine, then the second errors with > {error, closed}. > I don't know if I am misunderstanding something in SSL or Erlang. The > simplest client code I could come up with is: > run_test()-> > ? ? ssl:start(), > ? ? %%% works fine > ? ? {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, {keyfile, > "controller.key"} ]), > ? ? %%% this next one will return {error, closed} > ? ? {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, 0}, > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, {keyfile, > "controller.key"} ]). > > The python side gets the connection both times. Bizarrely, if I retry a > connection after getting {error, closed} it works. > Any insight? > Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From carlsson.richard@REDACTED Mon Jul 4 11:15:17 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 04 Jul 2011 11:15:17 +0200 Subject: [erlang-questions] Getting eunit to display the full term? In-Reply-To: References: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> Message-ID: <4E1184A5.4090100@gmail.com> On 07/04/2011 07:01 AM, Garrett Smith wrote: > On Sun, Jul 3, 2011 at 1:24 PM, Rudolph van Graan wrote: >> Is there a simple way (other than changing the source) to make eUnit display >> the entire term that didn't match in its output instead of the ... >> truncation? I find it pointless to have write a manual version of the test >> just to see what is not matching. > > It's not elegant, but I insert io:format("~p", [Result]) in places > where the result gets truncated. > > Output via io:format is suppressed when the test succeeds, suggesting > that this is somehow a feature. > > I would argue though that this is a design flaw in eunit -- results > should not be truncated. EUnit doesn't truncate the results at the source, but only in the tty output. There should probably be an option to the tty output module to control the truncation depth, but in general, you really want the tty output to be fairly short, or in many cases you'll just drown in thousands of lines of output per failed test. What's currently missing is a file logger backend, to write the complete (or as deep as you want) output to a file. It shouldn't be hard to make, if anyone has the time: copy the module eunit_tty and adapt it a bit, then hook it in as a standard option in the main eunit module. While developing, you can hook in your module using the option {report, X} where X = {Mod, Opts} for a listener module Mod called as Mod:start(Opts), or X = Pid/Atom for an existing listener process. See also the docs for eunit_surefire, which is a similar but slightly more complicated plugin. And of course, if anyone has ideas for other kinds of listeners, go ahead - it's not hard to make one. Sorry that I haven't had time to document the callback interface yet, but it's considered stable now. See the eunit_surefire docs as an example. /Richard From mrtndimitrov@REDACTED Mon Jul 4 12:31:42 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Mon, 04 Jul 2011 13:31:42 +0300 Subject: [erlang-questions] process priority Message-ID: <4E11968E.6010402@gmail.com> Hi, we have implemented a logging mechanism that calls gen_event:notify which returns immediately but nevertheless when performing extensive logging some genserver calls time out. I came to the conclusion that the erlang process manager is giving more time to the logging process and thus other processes never receive a chance to handle their queues. Is this conclusion possible or to seek the reason for the timeouts somewhere else? And if *yes* is it possible to give the logging process less priority? Thanks, Martin From klas.johansson@REDACTED Mon Jul 4 13:06:43 2011 From: klas.johansson@REDACTED (Klas Johansson) Date: Mon, 4 Jul 2011 13:06:43 +0200 Subject: [erlang-questions] Getting eunit to display the full term? In-Reply-To: <4E1184A5.4090100@gmail.com> References: <270FC5C5-4A39-4AED-A674-9772921A8D35@me.com> <4E1184A5.4090100@gmail.com> Message-ID: Hi, On Mon, Jul 4, 2011 at 11:15 AM, Richard Carlsson wrote: > On 07/04/2011 07:01 AM, Garrett Smith wrote: >> >> On Sun, Jul 3, 2011 at 1:24 PM, Rudolph van Graan >> ?wrote: >>> >>> Is there a simple way (other than changing the source) to make eUnit >>> display >>> the entire term that didn't match in its output instead of the ... >>> truncation? I find it pointless to have write a manual version of the >>> test >>> just to see what is not matching. >> >> It's not elegant, but I insert io:format("~p", [Result]) in places >> where the result gets truncated. >> >> Output via io:format is suppressed when the test succeeds, suggesting >> that this is somehow a feature. >> >> I would argue though that this is a design flaw in eunit -- results >> should not be truncated. > > EUnit doesn't truncate the results at the source, but only in the tty > output. There should probably be an option to the tty output module to > control the truncation depth, but in general, you really want the tty output > to be fairly short, or in many cases you'll just drown in thousands of lines > of output per failed test. > > What's currently missing is a file logger backend, to write the complete (or > as deep as you want) output to a file. It shouldn't be hard to make, if > anyone has the time: copy the module eunit_tty and adapt it a bit, then hook > it in as a standard option in the main eunit module. While developing, you > can hook in your module using the option {report, X} where X = {Mod, Opts} > for a listener module Mod called as Mod:start(Opts), or X = Pid/Atom for an > existing listener process. See also the docs for eunit_surefire, which is a > similar but slightly more complicated plugin. > > And of course, if anyone has ideas for other kinds of listeners, go ahead - > it's not hard to make one. Sorry that I haven't had time to document the > callback interface yet, but it's considered stable now. See the > eunit_surefire docs as an example. To people who start looking at eunit_surefire: you might want to have a look at the version which is cooking on the `pu' branch, since the one in `dev' is slightly broken when it comes to handling groups of tests. https://github.com/erlang/otp/tree/pu/lib/eunit/src Cheers, Klas From pablo.platt@REDACTED Mon Jul 4 13:12:36 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 4 Jul 2011 04:12:36 -0700 (PDT) Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> Message-ID: <1309777956.27562.YahooMailNeo@web112606.mail.gq1.yahoo.com> exprecs looks interesting. Not sure how hard it will be to patch erly_dtl to use it. Thanks ----- Original Message ----- From: Tim Watson To: Pablo Platt Cc: "erlang-questions@REDACTED" Sent: Monday, July 4, 2011 10:53 AM Subject: Re: [erlang-questions] parameterized modules alternative in chicago boss like use case > Alternative to pmods in that case could be records, proplist/dict, gen_server. > > - records > > To use a record, we need to tell the template about the record type. > {% for p#product in products %} > That's fine but we can only use the same template for several model types > if the records of all product types have the same structure: > {product_type1, {name, price, some_field}} > {product_type2, {name, price, another_field}} > We can't 'extend' a record in erlang so we'll have to manually make write the structure of all related records which will be hard in a large application. > You could extend (or rewrite) something like https://github.com/esl/parse_trans/blob/master/doc/exprecs.md to export a bunch of functions associated with the record. This stuff works at compile time (using parse transforms) but could be achieved at runtime with a bit of beam munging. From ulf.wiger@REDACTED Mon Jul 4 13:19:15 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 4 Jul 2011 13:19:15 +0200 Subject: [erlang-questions] process priority In-Reply-To: <4E11968E.6010402@gmail.com> References: <4E11968E.6010402@gmail.com> Message-ID: <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> I assume the logging process writes to disk? Are you running with a thread pool? If there is a lot of disk I/O, this can cause the emulator loop to block, unless there are enough I/O threads available. I'm guessing the logging process has 'normal' priority right now? It is possible to set the priority of a gen_event process, e.g. by installing a handler (which gets to execute in the process context, thus being able to call process_flag(priority, P)). However, running a process at 'low' priority is generally not a good idea. It could cause the logging process to fall too much behind, and eventually running out of memory, or it could even in some cases lead to priority inversion, if the number of runnable normal-priority processes is significantly greater than the number of low-priority processes. The runtime system does have one "feature", where it punishes the sender with extra reductions if the receiver's message queue is large. This could contribute to giving the logger process relatively more time than the other processes. Fundamentally, though, you have to ensure that you don't produce more log output than your I/O subsystem can manage. BR, Ulf W On 4 Jul 2011, at 12:31, Martin Dimitrov wrote: > Hi, > > we have implemented a logging mechanism that calls gen_event:notify > which returns immediately but nevertheless when performing extensive > logging some genserver calls time out. I came to the conclusion that the > erlang process manager is giving more time to the logging process and > thus other processes never receive a chance to handle their queues. > > Is this conclusion possible or to seek the reason for the timeouts > somewhere else? And if *yes* is it possible to give the logging process > less priority? > > Thanks, > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From mrtndimitrov@REDACTED Mon Jul 4 13:43:54 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Mon, 04 Jul 2011 14:43:54 +0300 Subject: [erlang-questions] process priority In-Reply-To: <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> Message-ID: <4E11A77A.3040504@gmail.com> Yes, the logging process writes to disk. I am not sure what thread pool means but our in our app only the logger does I/O operations. Can it block for more than 5 secs so the other processes to time out? On 7/4/2011 2:19 PM, Ulf Wiger wrote: > I assume the logging process writes to disk? > > Are you running with a thread pool? If there is a lot of disk I/O, this can cause the emulator loop to block, unless there are enough I/O threads available. > > I'm guessing the logging process has 'normal' priority right now? It is possible to set the priority of a gen_event process, e.g. by installing a handler (which gets to execute in the process context, thus being able to call process_flag(priority, P)). However, running a process at 'low' priority is generally not a good idea. It could cause the logging process to fall too much behind, and eventually running out of memory, or it could even in some cases lead to priority inversion, if the number of runnable normal-priority processes is significantly greater than the number of low-priority processes. > > The runtime system does have one "feature", where it punishes the sender with extra reductions if the receiver's message queue is large. This could contribute to giving the logger process relatively more time than the other processes. > > Fundamentally, though, you have to ensure that you don't produce more log output than your I/O subsystem can manage. > > BR, > Ulf W > > On 4 Jul 2011, at 12:31, Martin Dimitrov wrote: > >> Hi, >> >> we have implemented a logging mechanism that calls gen_event:notify >> which returns immediately but nevertheless when performing extensive >> logging some genserver calls time out. I came to the conclusion that the >> erlang process manager is giving more time to the logging process and >> thus other processes never receive a chance to handle their queues. >> >> Is this conclusion possible or to seek the reason for the timeouts >> somewhere else? And if *yes* is it possible to give the logging process >> less priority? >> >> Thanks, >> >> Martin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From mazen.harake@REDACTED Mon Jul 4 13:53:55 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 4 Jul 2011 13:53:55 +0200 Subject: [erlang-questions] process priority In-Reply-To: <4E11A77A.3040504@gmail.com> References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> Message-ID: My 2 cents. Generally, writing straight to disk is a bad thing. You should have a table (ets/mnesia) where you write your log lines and a process that periodically flushes it to disk. Depending on your system load this will help immensely. In very rare cases you would tweak the process priorities and if you do you should consider if your solution is "wrong" rather than the priority being the bad guy. GL HF /M On 4 July 2011 13:43, Martin Dimitrov wrote: > Yes, the logging process writes to disk. > I am not sure what thread pool means but our in our app only the logger > does I/O operations. Can it block for more than 5 secs so the other > processes to time out? > > On 7/4/2011 2:19 PM, Ulf Wiger wrote: >> I assume the logging process writes to disk? >> >> Are you running with a thread pool? If there is a lot of disk I/O, this can cause the emulator loop to block, unless there are enough I/O threads available. >> >> I'm guessing the logging process has 'normal' priority right now? It is possible to set the priority of a gen_event process, e.g. by installing a handler (which gets to execute in the process context, thus being able to call process_flag(priority, P)). However, running a process at 'low' priority is generally not a good idea. It could cause the logging process to fall too much behind, and eventually running out of memory, or it could even in some cases lead to priority inversion, if the number of runnable normal-priority processes is significantly greater than the number of low-priority processes. >> >> The runtime system does have one "feature", where it punishes the sender with extra reductions if the receiver's message queue is large. This could contribute to giving the logger process relatively more time than the other processes. >> >> Fundamentally, though, you have to ensure that you don't produce more log output than your I/O subsystem can manage. >> >> BR, >> Ulf W >> >> On 4 Jul 2011, at 12:31, Martin Dimitrov wrote: >> >>> Hi, >>> >>> we have implemented a logging mechanism that calls gen_event:notify >>> which returns immediately but nevertheless when performing extensive >>> logging some genserver calls time out. I came to the conclusion that the >>> erlang process manager is giving more time to the logging process and >>> thus other processes never receive a chance to handle their queues. >>> >>> Is this conclusion possible or to seek the reason for the timeouts >>> somewhere else? And if *yes* is it possible to give the logging process >>> less priority? >>> >>> Thanks, >>> >>> Martin >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Mon Jul 4 14:31:22 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 4 Jul 2011 14:31:22 +0200 Subject: [erlang-questions] process priority In-Reply-To: <4E11A77A.3040504@gmail.com> References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> Message-ID: <385437A7-DFC4-4487-B40F-D8E4A34F25D7@erlang-solutions.com> The thread pool is controlled by the +A flag, as in e.g. 'erl +A 256 ?' From the 'erts' manual: "+A size Sets the number of threads in async thread pool, valid range is 0-1024. Default is 0." In systems with a lot of disk IO, if your shell tends to become unresponsive, or some processes experience mysterious timeouts, it's a good idea to look at the thread pool. If you have too few threads, IO operations will run directly from the scheduler thread and block that scheduler. BR, Ulf W On 4 Jul 2011, at 13:43, Martin Dimitrov wrote: > Yes, the logging process writes to disk. > I am not sure what thread pool means but our in our app only the logger > does I/O operations. Can it block for more than 5 secs so the other > processes to time out? > > On 7/4/2011 2:19 PM, Ulf Wiger wrote: >> I assume the logging process writes to disk? >> >> Are you running with a thread pool? If there is a lot of disk I/O, this can cause the emulator loop to block, unless there are enough I/O threads available. >> >> I'm guessing the logging process has 'normal' priority right now? It is possible to set the priority of a gen_event process, e.g. by installing a handler (which gets to execute in the process context, thus being able to call process_flag(priority, P)). However, running a process at 'low' priority is generally not a good idea. It could cause the logging process to fall too much behind, and eventually running out of memory, or it could even in some cases lead to priority inversion, if the number of runnable normal-priority processes is significantly greater than the number of low-priority processes. >> >> The runtime system does have one "feature", where it punishes the sender with extra reductions if the receiver's message queue is large. This could contribute to giving the logger process relatively more time than the other processes. >> >> Fundamentally, though, you have to ensure that you don't produce more log output than your I/O subsystem can manage. >> >> BR, >> Ulf W >> >> On 4 Jul 2011, at 12:31, Martin Dimitrov wrote: >> >>> Hi, >>> >>> we have implemented a logging mechanism that calls gen_event:notify >>> which returns immediately but nevertheless when performing extensive >>> logging some genserver calls time out. I came to the conclusion that the >>> erlang process manager is giving more time to the logging process and >>> thus other processes never receive a chance to handle their queues. >>> >>> Is this conclusion possible or to seek the reason for the timeouts >>> somewhere else? And if *yes* is it possible to give the logging process >>> less priority? >>> >>> Thanks, >>> >>> Martin >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Mon Jul 4 15:27:01 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 4 Jul 2011 14:27:01 +0100 Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: <1309777956.27562.YahooMailNeo@web112606.mail.gq1.yahoo.com> References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> <1309777956.27562.YahooMailNeo@web112606.mail.gq1.yahoo.com> Message-ID: On 4 July 2011 12:12, Pablo Platt wrote: > exprecs looks interesting. > Not sure how hard it will be to patch erly_dtl to use it. > Actually come to think of it, erlydtl will take a proplist. So with exprecs, you can work (in your code) with a record and then use the (auto) exported exprecs functions to convert this to a property list. I do something similar to this (for serialization purposes) here: https://github.com/hyperthunk/nodewatch/blob/master/dxcommon/src/dxcommon.erl#L46 From kostis@REDACTED Mon Jul 4 15:37:56 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 04 Jul 2011 16:37:56 +0300 Subject: [erlang-questions] spec declaration for single list parameter with fixed size In-Reply-To: References: Message-ID: <4E11C234.8020408@cs.ntua.gr> Jesper Louis Andersen wrote: > On Sun, Jul 3, 2011 at 11:58, Michael Richter wrote: >> On 3 July 2011 16:10, Roberto Ostinelli wrote: >>> a very stupid spec declaration question. how can you declare the specs of >>> a function which has a single list with multiple arguments? >>> >>> For instance, consider this function: >> >>> myfun([One, Two]) -> > > This is better since you then have a product type rather than a > recursive type on lists. It is much simpler to work with from a typing > perspective. Also note that you would not easily be able to type > ["hello", 5] in a statically typed language without resorting to > either some kind of type tagging, polymorphic variants or existential > types. > > My advice would be to alter the structure of the function such that it > is easier to type. It tends to be safer from a programming perspective > as well. Both Jesper and Michael have given very good advice on this topic, but for the record, I guess, let me add my two cents here. First of all, as others have mentioned, when grouping together a fixed number of items, tuples rather than lists should better be used. Not only does this make sense from a typing perspective, but for more than one item tuples are also more memory efficient and thus slightly faster. For example [One, Two] needs 4 words while {One, Two} needs 3. Coming back to the original question, it's pretty clear that what the type language does it to make a abstraction over all terms (which belong to some type) and thus cannot possibly express all programmer intentions accurately. For example, it is currently not possible to declare lists with a certain number of elements, partly due to the fact that the type language has decided to make [T] an alias for list(T). Even if this constraint was lifted (e.g. [T] stood for a one element list for items of type T, [T1,T2] for a two element list of items of type T1 and T2, etc.) the type language would not be able to express all programmer intentions (e.g. lists of an even number of elements, lists whose length is a prime number, lists where the middle element is 42, etc.). This is fundamental limitation of all type languages: no matter how expressive they become, there are things that cannot (conveniently) be expressed in them. Kostis From watson.timothy@REDACTED Mon Jul 4 16:12:58 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 4 Jul 2011 15:12:58 +0100 Subject: [erlang-questions] process priority In-Reply-To: References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> Message-ID: > My 2 cents. > > Generally, writing straight to disk is a bad thing. You should have a > table (ets/mnesia) where you write your log lines and a process that > periodically flushes it to disk. Depending on your system load this > will help immensely. You could also consider disk_log, which offers a halfway solution to some of these issues, writing to disk but not necessarily flushing immediately. Also Ulf's comments about tweaking the thread pool size are important, although it's worth baring in mind that this can affect some linked-in drivers that are using async threads as well. From ericbmerritt@REDACTED Mon Jul 4 16:15:42 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 4 Jul 2011 09:15:42 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: I can actually see a ton of uses for this. Not the least of which is security, I just did a deep security piece in an erlang app, and it turned out well, but I could see where something like this would have made it much more elegant. The only thing that would concern me (from looking at your implementation) is how much over head is added to process creation. I think thats probably the main unknown, there are probably some other things as well. Just out of curiosity does a group leader go away when a group dies? On Sat, Jul 2, 2011 at 7:38 PM, Yurii Rashkovskii wrote: > Hi, > > I had an idea to play with this long weekend and I want to share some > results and get some feedback. > > Basically, I was wondering whether it will be feasible to extend the > concept of group leaders beyond I/O so that you can build your own > primitives around this construct of inherited group leaders. It seemed > to me that tasks like implicit configuration, security groups and such > can benefit from having a more generalized group leadership mechanism. > > In the last two days I wrote a quick-n-dirty proof of concept > implementation (https://github.com/spawngrid/otp/tree/group_leader_scope), > it's pretty rough but seems to work. I also wrote an early draft of an > EEP for this feature > (https://github.com/spawngrid/eep/blob/scoped-group-leaders/eeps/eep-scoped-group-leaders.md). > > The EEP has some examples, and you can play with that PoC > implementation (group_leader_scope branch). It's not ready for the > prime time, though ? it lacks automated tests and the performance > implications have not yet been measured (although this should be fixed > soon). No optimizations were considered at this moment. > > The EEP draft itself has not been sent to EEP editors as I plan to > elaborate on motivations and possibly add more examples. > > Either way, please let me know if this is of any interest to anybody > and if you have any thoughts about it. > > Have a nice weekend, > Yurii. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fred.hebert@REDACTED Mon Jul 4 16:19:22 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Mon, 4 Jul 2011 10:19:22 -0400 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: <8FB81A21-7092-47AF-B830-54D03FC72FBE@erlang-solutions.com> I agree this might be quite useful. To me one of the interesting points has to be the possibility to write your own protocols similar to the IO one. I could see something like a web server or any other resource handler of the kind allowing a bunch of workers to pipe their output straight out to some socket, file descriptor or whatever. The abstractions put in place in front of the IO server could be used across other resources with potentially different semantics (and thus requiring a different protocol). This could be a nice step towards that. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-04, at 10:15 AM, Eric Merritt wrote: > I can actually see a ton of uses for this. Not the least of which is > security, I just did a deep security piece in an erlang app, and it > turned out well, but I could see where something like this would have > made it much more elegant. The only thing that would concern me (from > looking at your implementation) is how much over head is added to > process creation. I think thats probably the main unknown, there are > probably some other things as well. > > Just out of curiosity does a group leader go away when a group dies? > On Sat, Jul 2, 2011 at 7:38 PM, Yurii Rashkovskii wrote: >> Hi, >> >> I had an idea to play with this long weekend and I want to share some >> results and get some feedback. >> >> Basically, I was wondering whether it will be feasible to extend the >> concept of group leaders beyond I/O so that you can build your own >> primitives around this construct of inherited group leaders. It seemed >> to me that tasks like implicit configuration, security groups and such >> can benefit from having a more generalized group leadership mechanism. >> >> In the last two days I wrote a quick-n-dirty proof of concept >> implementation (https://github.com/spawngrid/otp/tree/group_leader_scope), >> it's pretty rough but seems to work. I also wrote an early draft of an >> EEP for this feature >> (https://github.com/spawngrid/eep/blob/scoped-group-leaders/eeps/eep-scoped-group-leaders.md). >> >> The EEP has some examples, and you can play with that PoC >> implementation (group_leader_scope branch). It's not ready for the >> prime time, though ? it lacks automated tests and the performance >> implications have not yet been measured (although this should be fixed >> soon). No optimizations were considered at this moment. >> >> The EEP draft itself has not been sent to EEP editors as I plan to >> elaborate on motivations and possibly add more examples. >> >> Either way, please let me know if this is of any interest to anybody >> and if you have any thoughts about it. >> >> Have a nice weekend, >> Yurii. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Mon Jul 4 17:11:17 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 4 Jul 2011 17:11:17 +0200 Subject: [erlang-questions] Erlang Web Servers challenge Message-ID: Hi guys, I'd like your help for this small Web servers challenge. The idea is ver simple: measure "how fast" you can serve a "100 bytes" file (attached as "small.html") in "less than 01 second" by using one of the following web servers: - Yaws (http://yaws.hyber.org/download/yaws-1.90.tar.gz) - Mochiweb (https://github.com/mochi/mochiweb) - Misultin (https://github.com/ostinelli/misultin) - Cowboy (https://github.com/extend/cowboy) - Bare metal web server using "gen_tcp" The rules are very simple : 1. The first time you served the file, you MUST read it from disk. All subsequent requests to that file can use any caching mechanism! 2. Any (hidden) trick is allowed (eg. prim_inet, SMP enabled ...) 3. Erlang versions R14B to R14B03 4. The HTTP client to use for this benchmark needs to be "Apache AB" (http://httpd.apache.org/docs/2.0/programs/ab.html) with: $ ab -n 200000 -c 100 -t1 -k "http://127.0.0.1:8080/small.html" I hope that the authors if these web servers will also contribute to this bench as they know their software better than anybody else. Challenge's open ;-) Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Mon Jul 4 17:15:33 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 4 Jul 2011 17:15:33 +0200 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: <882B10B3-4EE6-430B-960E-5CA3658B072C@erlang-solutions.com> On 4 Jul 2011, at 16:15, Eric Merritt wrote: > I can actually see a ton of uses for this. Not the least of which is > security, I just did a deep security piece in an erlang app, and it > turned out well, but I could see where something like this would have > made it much more elegant How, exactly? The things that are difficult to do today, in terms of security in Erlang, is to *enforce* security in a way that cannot be subverted. Group leaders are natural for IO, since IO necessarily involves sending a message to some entity. Some other things are overloaded - e.g. the application controller using the group leader to find out which application a process belongs to. That is a clear case where I think this would be useful: an "application group leader". But for security, I don't see it solving any of the problems I consider difficult - perhaps this is my failing: after having implemented reasonably transparent security in ErlHive, I get all kinds of associations to nasty problems that wouldn't be helped by this suggestion. But perhaps you are viewing it from a different angle? BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From pablo.platt@REDACTED Mon Jul 4 17:32:31 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 4 Jul 2011 08:32:31 -0700 (PDT) Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> <1309777956.27562.YahooMailNeo@web112606.mail.gq1.yahoo.com> Message-ID: <1309793551.79962.YahooMailNeo@web112614.mail.gq1.yahoo.com> proplist will work but it won't allow me to do cool tricks like define a field value that instructs the erlydtl tag to auto load child models. For example, if we have the following record: {group_product, {"my-group-product", 15, 100, {#load, children, 15}} We might have a tag that knows how to load child products from the db. I'm trying to use exprecs as you suggested with a custom erlydtl tags module. The module: -module(mytags). -export([mytag/2]). mytag(Variables, RenderOptions) -> ??? io:format("~p~n", [Variables]), ??? io:format("~p~n", [RenderOptions]), ??? ["test"]. Template: {% mytag %} Compile with: erlydtl:compile("test.dtl", "test_dtl", [{out_dir, "./"}, {custom_tags_module, "mytags"}]). I don't understand how to pass arguments to the tag and how to have something like {% for a in l %} {{ a.attr }} {% endfor %} ----- Original Message ----- From: Tim Watson To: Pablo Platt Cc: "erlang-questions@REDACTED" Sent: Monday, July 4, 2011 4:27 PM Subject: Re: [erlang-questions] parameterized modules alternative in chicago boss like use case On 4 July 2011 12:12, Pablo Platt wrote: > exprecs looks interesting. > Not sure how hard it will be to patch erly_dtl to use it. > Actually come to think of it, erlydtl will take a proplist. So with exprecs, you can work (in your code) with a record and then use the (auto) exported exprecs functions to convert this to a property list. I do something similar to this (for serialization purposes) here: https://github.com/hyperthunk/nodewatch/blob/master/dxcommon/src/dxcommon.erl#L46 From essen@REDACTED Mon Jul 4 17:38:40 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 04 Jul 2011 17:38:40 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: <4E11DE80.1040803@dev-extend.eu> On 07/04/2011 05:11 PM, Zabrane Mickael wrote: > Hi guys, > > I'd like your help for this small Web servers challenge. > > The idea is ver simple: measure "how fast" you can serve a "100 bytes" > file (attached as "small.html") > in "less than 01 second" by using one of the following web servers: > > - Yaws (http://yaws.hyber.org/download/yaws-1.90.tar.gz) > - Mochiweb (https://github.com/mochi/mochiweb) > - Misultin (https://github.com/ostinelli/misultin) > - Cowboy (https://github.com/extend/cowboy) > - Bare metal web server using "gen_tcp" I don't really understand where the challenge is. Also yaws should be faster considering it supports sendfile whereas the others don't without much additional work beforehand. So regardless of the server being used you'd probably just be benchmarking sendfile and not the servers. -- Lo?c Hoguin Dev:Extend From ericbmerritt@REDACTED Mon Jul 4 17:47:48 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 4 Jul 2011 10:47:48 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: <882B10B3-4EE6-430B-960E-5CA3658B072C@erlang-solutions.com> References: <882B10B3-4EE6-430B-960E-5CA3658B072C@erlang-solutions.com> Message-ID: > On Mon, Jul 4, 2011 at 10:15 AM, Ulf Wiger wrote: > > On 4 Jul 2011, at 16:15, Eric Merritt wrote: > >> I can actually see a ton of uses for this. Not the least of which is >> security, I just did a deep security piece in an erlang app, and it >> turned out well, but I could see where something like this would have >> made it much more elegant > > How, exactly? In the existing model a context is passed through out the system, identifying the users and their capabilities. In this case, assuming that all threads related to use where spawned at some level by the incoming process, a context would not have had to be passed, it could have been implicit in the group leader. This would have been much less invasive in the system. I made no attempt to protect against developers adding malicious code to the system. That is hard regardless of the model. Perhaps this would have been better called an access control system rather then a security system given the assumptions. > > The things that are difficult to do today, in terms of security in Erlang, is to *enforce* security in a way that cannot be subverted. > > Group leaders are natural for IO, since IO necessarily involves sending a message to some entity. Some other things are overloaded - >e.g. the application controller using the group leader to find out which application a process belongs to. That is a clear case where I think >this would be useful: an "application group leader". But for security, I don't see it solving any of the problems I consider difficult - perhaps >this is my failing: after having implemented reasonably transparent security in ErlHive, I get all kinds of associations to nasty problems >that wouldn't be helped by this suggestion. But perhaps you are viewing it from a different angle? > I am not thinking about this as solving any truly hard problems, not that can't be solved in some other way. It just would have made the code created for this project, more elegant, and the security aspects much less invasive then they currently are, at the cost of slightly more difficult testing and some knowledge about where the process group boundries are. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From yrashk@REDACTED Mon Jul 4 18:13:50 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 4 Jul 2011 09:13:50 -0700 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: Eric, > The only thing that would concern me (from > looking at your implementation) is how much over head is added to > process creation. I think thats probably the main unknown, there are > probably some other things as well. Judging by my limited "unscientific" performance testing that I did yesterday, I've got virtually no impact for situations where you do not create any extra group leaders (i.e. running code that can be done today). In a case when you have a hundred extra group leaders, it slows down by about 2.5 times. Please bear in mind that current implementation is a proof-of-concept level quality. I can already foresee some techniques for speeding up group leader allocation by allocating on process creation in bulk, it's rather simple and will reduce number of allocations to just one. In fact, this is so easy I might update this PoC implementation rather soon! It sounds like a good idea. > Just out of curiosity does a group leader go away when a group dies? Define group death? From ericbmerritt@REDACTED Mon Jul 4 18:18:39 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 4 Jul 2011 11:18:39 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: > > >> Just out of curiosity does a group leader go away when a group dies? > > Define group death? > When when all processes in a group have terminated, either normally or abnormally. From yrashk@REDACTED Mon Jul 4 18:21:41 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 4 Jul 2011 09:21:41 -0700 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 9:18 AM, Eric Merritt wrote: >> >> >>> Just out of curiosity does a group leader go away when a group dies? >> >> Define group death? >> > > When when all processes in a group have terminated, either normally or > abnormally. I have intentionally decided not to overcomplicate current implementation. Right now nothing will happen. From dgriff1@REDACTED Mon Jul 4 19:10:15 2011 From: dgriff1@REDACTED (Daniel Griffin) Date: Mon, 4 Jul 2011 12:10:15 -0500 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: The twisted python server defaults to accepting SSL 2 or 3. I am using erlang R14B01. Is there some rule saying that you can't have multiple SSL connections between 2 programs? What sort of timing issues could I be having? Thanks, Dan On Mon, Jul 4, 2011 at 4:00 AM, Ingela Andin wrote: > Hi! > > It sounds like you are experiencing some kind of timing problem. > I tried to write a test case to repeat it but I am not getting > any problem. Which ssl-version you are running? > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/7/2 Daniel Griffin : > > I am using SSL sockets to talk between an erlang program and a twisted > > python program. Different processes in my erlang app occasionally have to > > open connections and the first one opens fine, then the second errors > with > > {error, closed}. > > I don't know if I am misunderstanding something in SSL or Erlang. The > > simplest client code I could come up with is: > > run_test()-> > > ssl:start(), > > %%% works fine > > {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, > > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, > {keyfile, > > "controller.key"} ]), > > %%% this next one will return {error, closed} > > {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, 0}, > > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, > {keyfile, > > "controller.key"} ]). > > > > The python side gets the connection both times. Bizarrely, if I retry a > > connection after getting {error, closed} it works. > > Any insight? > > Dan > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Mon Jul 4 19:39:09 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 4 Jul 2011 18:39:09 +0100 Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> Message-ID: <20110704173909.GA16369@hanele> # Pablo Platt 2011-07-04: > The powerful part is being able to use the model in an erl_dtl template: > {% for greeting in greetings %} >
  • {{ greeting.text }}
    • > {% endfor %} [I'll probably be slightly offtopic, but this caught my attention.] Hmm, looks like a particularly complicated way to say: [{li, [], Greeting} || Greeting <- somehow_get_list_of_greetings()] And you'd pass that to a function that expands [{Tag, Attrs, Body}] to HTML (doing some quoting here and there). My point is: why do people keep inventing obscure ad-hoc languages instead of just encoding stuff as Erlang terms and having normal Erlang functions operate on them? Your other concerns then become the usual question of structuring your code properly, finding out appropriate representations of things, abstracting out common constructs etc. But then I must be missing something here; I always must be when it comes to Awesome Modern Intarweb Techniques. BR, -- Jachym From zabrane3@REDACTED Mon Jul 4 19:41:26 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 4 Jul 2011 19:41:26 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <4E11DE80.1040803@dev-extend.eu> References: <4E11DE80.1040803@dev-extend.eu> Message-ID: >> I don't really understand where the challenge is. The challenge as stated is very really simple. How fast you can serve a 100 bytes files with your web server "Cowboy" for example in less than a second. > Also yaws should be faster considering it supports sendfile whereas the > others don't without much additional work beforehand. So regardless of > the server being used you'd probably just be benchmarking sendfile and > not the servers. Don't suppose anything (sendfile) before benchmarking it Lo?c. From fred.hebert@REDACTED Mon Jul 4 19:43:03 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Mon, 4 Jul 2011 13:43:03 -0400 Subject: [erlang-questions] parameterized modules alternative in chicago boss like use case In-Reply-To: <20110704173909.GA16369@hanele> References: <1309759555.37030.YahooMailNeo@web112611.mail.gq1.yahoo.com> <20110704173909.GA16369@hanele> Message-ID: <41A1E20A-D25F-431A-B765-0902CED8C18A@erlang-solutions.com> Encoding stuff as Erlang terms means that if you're working with a designer, integrator, a front-end dev or just any developer who doesn't know about Erlang, you're now forcing them to understand Erlang to work on your site, rather than letting them stick to the technology stacks they already know. To be on topic, I'm more likely to stick to dicts or gb_trees when dealing with ErlyDTL, and sometimes proplists when the semantics lend themselves to that. In most cases [that I encounter] these structures are good enough to handle everything efficiently. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-04, at 13:39 PM, Jachym Holecek wrote: > # Pablo Platt 2011-07-04: >> The powerful part is being able to use the model in an erl_dtl template: >> {% for greeting in greetings %} >>
  • {{ greeting.text }}
    • >> {% endfor %} > > [I'll probably be slightly offtopic, but this caught my attention.] > > Hmm, looks like a particularly complicated way to say: > > [{li, [], Greeting} || Greeting <- somehow_get_list_of_greetings()] > > And you'd pass that to a function that expands [{Tag, Attrs, Body}] to > HTML (doing some quoting here and there). My point is: why do people keep > inventing obscure ad-hoc languages instead of just encoding stuff as > Erlang terms and having normal Erlang functions operate on them? > > Your other concerns then become the usual question of structuring your code > properly, finding out appropriate representations of things, abstracting out > common constructs etc. > > But then I must be missing something here; I always must be when it comes > to Awesome Modern Intarweb Techniques. > > BR, > -- Jachym > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Mon Jul 4 19:52:47 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 04 Jul 2011 19:52:47 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: <4E11DE80.1040803@dev-extend.eu> Message-ID: <4E11FDEF.7000402@dev-extend.eu> On 07/04/2011 07:41 PM, Zabrane Mickael wrote: >>> I don't really understand where the challenge is. > > The challenge as stated is very really simple. > How fast you can serve a 100 bytes files with your web server "Cowboy" for example in less than a second. Only yaws is suited for this "challenge", the others aren't meant to serve files and would be better coupled with something like cherokee or nginx deployed on multiple servers each using separate domains. Even yaws with sendfile would probably be beaten by the C servers for that purpose. Though I guess measuring that one would be interesting. >> Also yaws should be faster considering it supports sendfile whereas the >> others don't without much additional work beforehand. So regardless of >> the server being used you'd probably just be benchmarking sendfile and >> not the servers. > > Don't suppose anything (sendfile) before benchmarking it Lo?c. I don't suppose anything. Kernel cache is faster than userland cache. -- Lo?c Hoguin Dev:Extend From freza@REDACTED Mon Jul 4 19:56:15 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 4 Jul 2011 18:56:15 +0100 Subject: [erlang-questions] process priority In-Reply-To: References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> Message-ID: <20110704175615.GB16369@hanele> # Mazen Harake 2011-07-04: > My 2 cents. > > Generally, writing straight to disk is a bad thing. You should have a > table (ets/mnesia) where you write your log lines and a process that > periodically flushes it to disk. Depending on your system load this > will help immensely. The other way around actually! :) Pass log items to gen_server in synchronous calls, that gives you end-to-end flow control. Use file in raw mode, leverage delayed_write flag. Make sure your gen_server has minimal processing overhead. Any kind of per-item processing in the server is a clear no-go, do all formatting in caller's context. It helps to do iolist_to_binary/1 in some carefully chosen callsites -- but you won't need that, following the above principles will give you a fast-enough solution (sustained load of >50k messages per second -- iolists about 100B in size -- is no problem, I've seen something like 70k peak with our logging library). Of course you don't want to have one logger process for the whole system, instead make it easy for each application to open as many as it needs for its audit logs, event logs, trace logs and such. Using ETS is of course doable, but locking overhead is about the same or higher than with plain message passing, you lose all flow control thus risking memory explosion (I have a nice graph handy demonstrating that, but probably can't publish it), and you copy every messsage twice instead of just once. (I don't have measurements on the locking and copying stuff, it's my recollection of reading relevant bits of ERTS -- could be wrong.) > In very rare cases you would tweak the process priorities and if you > do you should consider if your solution is "wrong" rather than the > priority being the bad guy. I agree with this. BR, -- Jachym From zabrane3@REDACTED Mon Jul 4 20:24:03 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 4 Jul 2011 20:24:03 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <4E11FDEF.7000402@dev-extend.eu> References: <4E11DE80.1040803@dev-extend.eu> <4E11FDEF.7000402@dev-extend.eu> Message-ID: >>> Also yaws should be faster considering it supports sendfile whereas the >>> others don't without much additional work beforehand. So regardless of >>> the server being used you'd probably just be benchmarking sendfile and >>> not the servers. >> >> Don't suppose anything (sendfile) before benchmarking it Lo?c. > > I don't suppose anything. Kernel cache is faster than userland cache. I don't think so. I'm using "sendfile" (extracted from yaws) https://github.com/tuncer/sendfile/commits/master/ (thanks to Steve and Tuncer) in our production app server for while now. Except a noticed decrease in CPU usage, it didn't make huge difference compared to "gen_tcp:send" in term of speed. But again, that's not fair because our app server is very special. I really hope you can fine tune Cowboy (if possible) and let us know how fast can it serve a 100 bytes file in less than a second. Thanks Lo?c! N.B: for the other web servers authors, please take 10mn for the challenge! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Jul 4 23:17:18 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 4 Jul 2011 23:17:18 +0200 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: Hi again! 2011/7/4 Daniel Griffin : > The twisted python server defaults to accepting SSL 2 or 3. I am using > erlang?R14B01. Is there some rule saying that you can't have multiple SSL > connections between 2 programs? No, it should not be a problem. > What sort of timing issues could I be having? I will investigate tomorrow if there could be some sort of timing issue. I am not saying there is, I am just saying your description of what is happening makes me think of strange timing issues, (even though I can not think of a reason there should be sush an isssue at the moment) but I had very little to go on. Knowing which version of the code you are running makes it easier. If you have the possibility to try the latest version of the code or atleast R14B03 that would be good. Regards Ingela Erlang/OTP team - Ericsson AB > Thanks, > Dan > On Mon, Jul 4, 2011 at 4:00 AM, Ingela Andin wrote: >> >> Hi! >> >> It sounds like you are experiencing some kind of timing problem. >> I tried to write a test case to repeat it but I am not getting >> any problem. Which ssl-version you are running? >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/7/2 Daniel Griffin : >> > I am using SSL sockets to talk between an erlang program and a twisted >> > python program. Different processes in my erlang app occasionally have >> > to >> > open connections and the first one opens fine, then the second errors >> > with >> > {error, closed}. >> > I don't know if I am misunderstanding something in SSL or Erlang. The >> > simplest client code I could come up with is: >> > run_test()-> >> > ? ? ssl:start(), >> > ? ? %%% works fine >> > ? ? {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, >> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >> > {keyfile, >> > "controller.key"} ]), >> > ? ? %%% this next one will return {error, closed} >> > ? ? {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, 0}, >> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >> > {keyfile, >> > "controller.key"} ]). >> > >> > The python side gets the connection both times. Bizarrely, if I retry a >> > connection after getting {error, closed} it works. >> > Any insight? >> > Dan >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > From ok@REDACTED Tue Jul 5 01:14:24 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 5 Jul 2011 11:14:24 +1200 Subject: [erlang-questions] spec declaration for single list parameter with fixed size In-Reply-To: References: Message-ID: <72FCB9FD-1029-4A45-A1F0-EA146E4D6D01@cs.otago.ac.nz> On 3/07/2011, at 8:10 PM, Roberto Ostinelli wrote: > dear list, > > a very stupid spec declaration question. how can you declare the specs of a function which has a single list with multiple arguments? ... > is there a better way to do so? Don't (ab)use a list like this. If the list always has two elements, why not make them separate arguments? If the elements are fixed in number and differing in type, why is this a list and not a tuple? From robert.virding@REDACTED Tue Jul 5 01:57:53 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 4 Jul 2011 23:57:53 +0000 (GMT) Subject: [erlang-questions] Scoped group leaders In-Reply-To: Message-ID: <668896959.52301309823873219.JavaMail.root@zimbra> ----- "Eric Merritt" wrote: > > > > > >> Just out of curiosity does a group leader go away when a group > dies? > > > > Define group death? > > > > When when all processes in a group have terminated, either normally > or > abnormally. Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader. Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat. Robert From yrashk@REDACTED Tue Jul 5 02:01:07 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 4 Jul 2011 17:01:07 -0700 Subject: [erlang-questions] Scoped group leaders In-Reply-To: <668896959.52301309823873219.JavaMail.root@zimbra> References: <668896959.52301309823873219.JavaMail.root@zimbra> Message-ID: On Mon, Jul 4, 2011 at 4:57 PM, Robert Virding wrote: > Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader. > > Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat. That's right. I personally understand them as "ephemeral" notions, just like you defined. That's why I don't do anything if a group leader dies. It's a duty of individual process to decide what to do with this. Yurii From roberto@REDACTED Tue Jul 5 09:05:30 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 5 Jul 2011 09:05:30 +0200 Subject: [erlang-questions] spec declaration for single list parameter with fixed size In-Reply-To: <4E11C234.8020408@cs.ntua.gr> References: <4E11C234.8020408@cs.ntua.gr> Message-ID: 2011/7/4 Kostis Sagonas > Jesper Louis Andersen wrote: > >> On Sun, Jul 3, 2011 at 11:58, Michael Richter >> wrote: >> >>> On 3 July 2011 16:10, Roberto Ostinelli wrote: >>> >>>> a very stupid spec declaration question. how can you declare the specs >>>> of >>>> a function which has a single list with multiple arguments? >>>> >>>> For instance, consider this function: >>>> >>> >>> myfun([One, Two]) -> >>>> >>> >> This is better since you then have a product type rather than a >> recursive type on lists. It is much simpler to work with from a typing >> perspective. Also note that you would not easily be able to type >> ["hello", 5] in a statically typed language without resorting to >> either some kind of type tagging, polymorphic variants or existential >> types. >> >> My advice would be to alter the structure of the function such that it >> is easier to type. It tends to be safer from a programming perspective >> as well. >> > > Both Jesper and Michael have given very good advice on this topic, but for > the record, I guess, let me add my two cents here. > > First of all, as others have mentioned, when grouping together a fixed > number of items, tuples rather than lists should better be used. Not only > does this make sense from a typing perspective, but for more than one item > tuples are also more memory efficient and thus slightly faster. For example > [One, Two] needs 4 words while {One, Two} needs 3. > > Coming back to the original question, it's pretty clear that what the type > language does it to make a abstraction over all terms (which belong to some > type) and thus cannot possibly express all programmer intentions accurately. > For example, it is currently not possible to declare lists with a certain > number of elements, partly due to the fact that the type language has > decided to make [T] an alias for list(T). Even if this constraint was lifted > (e.g. [T] stood for a one element list for items of type T, [T1,T2] for a > two element list of items of type T1 and T2, etc.) the type language would > not be able to express all programmer intentions (e.g. lists of an even > number of elements, lists whose length is a prime number, lists where the > middle element is 42, etc.). This is fundamental limitation of all type > languages: no matter how expressive they become, there are things that > cannot (conveniently) be expressed in them. > > Kostis > Jesper and Michael, thank you for your feedback on this. I am aware of the necessity of using tuples in this condition instead of lists. However, I was using a library whose callback function is getting arguments as list. Kostis, thank you for clarifying this. Cheers, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mazen.harake@REDACTED Tue Jul 5 09:57:30 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 5 Jul 2011 09:57:30 +0200 Subject: [erlang-questions] process priority In-Reply-To: <20110704175615.GB16369@hanele> References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> <20110704175615.GB16369@hanele> Message-ID: Writing to mnesia/dets requires no locking, nothing mentionable anyway, since the messages (internal ones) are serialized just like your gen_server will have its messages serialized (also, use dirty-operations). This creates end to end flow control, actually even more so because you won't need the extra process (your logging process) in between the write. Perhaps your small 100B messages work so well with delayed_write because you can write many of them to memory before they are flushed to a file thus not hogging the disk, but I the bigger messages you have the larger your in memory size needs to be to to avoid this. Delayed write does of course work well but I have experience that says that writing and buffering it up in tables can be helpful to avoid disk thrashing when messages are large (or higher volume). I don't remember exactly how much throughput we had (and I don't want to guess since it will be mere speculation without having hard data) but it helped immensely. So I guess OP now have 2 suggestions which of course isn't bad ;) One should also keep in mind though that different situation may have different needs, would be interesting to see how they would measure up. /M On 4 July 2011 19:56, Jachym Holecek wrote: > # Mazen Harake 2011-07-04: >> My 2 cents. >> >> Generally, writing straight to disk is a bad thing. You should have a >> table (ets/mnesia) where you write your log lines and a process that >> periodically flushes it to disk. Depending on your system load this >> will help immensely. > > The other way around actually! :) Pass log items to gen_server in synchronous > calls, that gives you end-to-end flow control. Use file in raw mode, leverage > delayed_write flag. Make sure your gen_server has minimal processing overhead. > Any kind of per-item processing in the server is a clear no-go, do all > formatting in caller's context. It helps to do iolist_to_binary/1 in some > carefully chosen callsites -- but you won't need that, following the above > principles will give you a fast-enough solution (sustained load of >50k > messages per second -- iolists about 100B in size -- is no problem, I've seen > something like 70k peak with our logging library). Of course you don't want > to have one logger process for the whole system, instead make it easy for > each application to open as many as it needs for its audit logs, event logs, > trace logs and such. > > Using ETS is of course doable, but locking overhead is about the same or > higher than with plain message passing, you lose all flow control thus > risking memory explosion (I have a nice graph handy demonstrating that, > but probably can't publish it), and you copy every messsage twice instead > of just once. (I don't have measurements on the locking and copying stuff, > it's my recollection of reading relevant bits of ERTS -- could be wrong.) > >> In very rare cases you would tweak the process priorities and if you >> do you should consider if your solution is "wrong" rather than the >> priority being the bad guy. > > I agree with this. > > BR, > ? ? ? ?-- Jachym > From rvg.mailing@REDACTED Tue Jul 5 11:02:02 2011 From: rvg.mailing@REDACTED (Rudolph van Graan) Date: Tue, 05 Jul 2011 10:02:02 +0100 Subject: [erlang-questions] ODBC driver under Solaris 10 Message-ID: <2DE0AFF0-4E49-487B-8CA5-1A004341A153@me.com> Hi there, Does someone know the exact configuration (combination of compilers and packages) that is needed to get Erlang's ODBC driver running under Solaris? ********************** APPLICATIONS DISABLED ********************** ********************************************************************* odbc : ODBC library - link check failed ********************************************************************* checking for void *... yes checking size of void *... 4 checking for odbc in standard locations... -L/usr/local/lib checking for SQLAllocHandle in -lodbc... no configure: WARNING: "ODBC library - link check failed" configure: creating ./config.status config.status: creating c_src/i386-pc-solaris2.10/Makefile The odbc driver is present (it is unixodbc 2.30 compiled with the same tool chain as Erlang): -rwxr-xr-x 1 root root 942 Jul 1 14:25 /usr/local/lib/libodbc.la lrwxrwxrwx 1 root root 16 Jul 1 14:25 /usr/local/lib/libodbc.so -> libodbc.so.1.0.0 lrwxrwxrwx 1 root root 16 Jul 1 14:25 /usr/local/lib/libodbc.so.1 -> libodbc.so.1.0.0 -rwxr-xr-x 1 root root 1688300 Jul 1 14:25 /usr/local/lib/libodbc.so.1.0.0 -rwxr-xr-x 1 root root 954 Jul 1 14:25 /usr/local/lib/libodbccr.la lrwxrwxrwx 1 root root 18 Jul 1 14:25 /usr/local/lib/libodbccr.so -> libodbccr.so.1.0.0 lrwxrwxrwx 1 root root 18 Jul 1 14:25 /usr/local/lib/libodbccr.so.1 -> libodbccr.so.1.0.0 -rwxr-xr-x 1 root root 682132 Jul 1 14:25 /usr/local/lib/libodbccr.so.1.0.0 -rwxr-xr-x 1 root root 966 Jul 1 14:25 /usr/local/lib/libodbcinst.la lrwxrwxrwx 1 root root 20 Jul 1 14:25 /usr/local/lib/libodbcinst.so -> libodbcinst.so.1.0.0 lrwxrwxrwx 1 root root 20 Jul 1 14:25 /usr/local/lib/libodbcinst.so.1 -> libodbcinst.so.1.0.0 -rwxr-xr-x 1 root root 321176 Jul 1 14:25 /usr/local/lib/libodbcinst.so.1.0.0 Any ideas? Rudolph -------------- next part -------------- An HTML attachment was scrubbed... URL: From valentin@REDACTED Tue Jul 5 12:24:36 2011 From: valentin@REDACTED (Valentin Micic) Date: Tue, 5 Jul 2011 12:24:36 +0200 Subject: [erlang-questions] ODBC driver under Solaris 10 In-Reply-To: <2DE0AFF0-4E49-487B-8CA5-1A004341A153@me.com> References: <2DE0AFF0-4E49-487B-8CA5-1A004341A153@me.com> Message-ID: Just to cove the basics... do you have LIBPATH configured to cover this path as well? V/ On 05 Jul 2011, at 11:02 AM, Rudolph van Graan wrote: > Hi there, > > Does someone know the exact configuration (combination of compilers and packages) that is needed to get Erlang's ODBC driver running under Solaris? > > ********************** APPLICATIONS DISABLED ********************** > ********************************************************************* > > odbc : ODBC library - link check failed > > ********************************************************************* > > checking for void *... yes > checking size of void *... 4 > checking for odbc in standard locations... -L/usr/local/lib > checking for SQLAllocHandle in -lodbc... no > configure: WARNING: "ODBC library - link check failed" > configure: creating ./config.status > config.status: creating c_src/i386-pc-solaris2.10/Makefile > > The odbc driver is present (it is unixodbc 2.30 compiled with the same tool chain as Erlang): > > -rwxr-xr-x 1 root root 942 Jul 1 14:25 /usr/local/lib/libodbc.la > lrwxrwxrwx 1 root root 16 Jul 1 14:25 /usr/local/lib/libodbc.so -> libodbc.so.1.0.0 > lrwxrwxrwx 1 root root 16 Jul 1 14:25 /usr/local/lib/libodbc.so.1 -> libodbc.so.1.0.0 > -rwxr-xr-x 1 root root 1688300 Jul 1 14:25 /usr/local/lib/libodbc.so.1.0.0 > -rwxr-xr-x 1 root root 954 Jul 1 14:25 /usr/local/lib/libodbccr.la > lrwxrwxrwx 1 root root 18 Jul 1 14:25 /usr/local/lib/libodbccr.so -> libodbccr.so.1.0.0 > lrwxrwxrwx 1 root root 18 Jul 1 14:25 /usr/local/lib/libodbccr.so.1 -> libodbccr.so.1.0.0 > -rwxr-xr-x 1 root root 682132 Jul 1 14:25 /usr/local/lib/libodbccr.so.1.0.0 > -rwxr-xr-x 1 root root 966 Jul 1 14:25 /usr/local/lib/libodbcinst.la > lrwxrwxrwx 1 root root 20 Jul 1 14:25 /usr/local/lib/libodbcinst.so -> libodbcinst.so.1.0.0 > lrwxrwxrwx 1 root root 20 Jul 1 14:25 /usr/local/lib/libodbcinst.so.1 -> libodbcinst.so.1.0.0 > -rwxr-xr-x 1 root root 321176 Jul 1 14:25 /usr/local/lib/libodbcinst.so.1.0.0 > > Any ideas? > > Rudolph > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuleshovmail@REDACTED Tue Jul 5 14:27:01 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Tue, 5 Jul 2011 12:27:01 +0000 Subject: [erlang-questions] function badarg Message-ID: Hello, I have loop/2 function. This is main cycle of processing input data. Approximate form of this function: loop(Session, State) -> {1} -> ..... loop(Session, State); {2} -> ..... loop(Session, State); {3} -> ..... loop(Session, State); _ -> loop(Session, State). In fact, this function big. Something about 1000 string of code. Sometimes this function crash with: Error in process <0.384.0> on node 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} Is there instruments or methods to know where it occurs and how it can catch this? Thank you. From demeshchuk@REDACTED Tue Jul 5 14:28:29 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Tue, 5 Jul 2011 16:28:29 +0400 Subject: [erlang-questions] function badarg In-Reply-To: References: Message-ID: Use proc_lib:spawn_link instead of erlang:spawn_link. That's possibly the easiest way. On Tue, Jul 5, 2011 at 4:27 PM, Alexander Kuleshov wrote: > Hello, > > I have loop/2 function. This is main cycle of processing input data. > > Approximate form of this function: > > loop(Session, State) -> > ?{1} -> > ? ? ? ?..... > ? ? ? loop(Session, State); > {2} -> > ? ? ? ?..... > ? ? ? loop(Session, State); > {3} -> > ? ? ? ?..... > ? ? ? loop(Session, State); > _ -> > ? ? loop(Session, State). > > In fact, this function big. Something about 1000 string of code. > > Sometimes this function crash with: Error in process <0.384.0> on node > 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} > > Is there instruments or methods to know where it occurs and how it can > catch this? > > Thank you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From ingela.andin@REDACTED Tue Jul 5 15:12:00 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 5 Jul 2011 15:12:00 +0200 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: Hi! I have not actually pinpointed the problem but I got my test-case to fail on R14B01 but runs fine on R14B02 and R1B03 and on the main development branch. Hence upgrade and let me know how it goes. Regards Ingela Erlang/OTP team - Ericsson AB 2011/7/4 Ingela Andin : > Hi again! > > 2011/7/4 Daniel Griffin : >> The twisted python server defaults to accepting SSL 2 or 3. I am using >> erlang?R14B01. Is there some rule saying that you can't have multiple SSL >> connections between 2 programs? > > No, it should not be a problem. > >> What sort of timing issues could I be having? > > I will investigate tomorrow if there could be some sort of timing > issue. I am not saying > there is, I am ?just saying your description of what is happening > makes me think of strange timing issues, (even though I can not think > of a reason there should be sush an isssue at the moment) but I had > very little to go on. Knowing which version of the code you are > running makes it easier. ?If you have the possibility to try the > latest version of the code or atleast R14B03 > that would be good. > > Regards Ingela Erlang/OTP team - Ericsson AB > >> Thanks, >> Dan >> On Mon, Jul 4, 2011 at 4:00 AM, Ingela Andin wrote: >>> >>> Hi! >>> >>> It sounds like you are experiencing some kind of timing problem. >>> I tried to write a test case to repeat it but I am not getting >>> any problem. Which ssl-version you are running? >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> 2011/7/2 Daniel Griffin : >>> > I am using SSL sockets to talk between an erlang program and a twisted >>> > python program. Different processes in my erlang app occasionally have >>> > to >>> > open connections and the first one opens fine, then the second errors >>> > with >>> > {error, closed}. >>> > I don't know if I am misunderstanding something in SSL or Erlang. The >>> > simplest client code I could come up with is: >>> > run_test()-> >>> > ? ? ssl:start(), >>> > ? ? %%% works fine >>> > ? ? {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >>> > {keyfile, >>> > "controller.key"} ]), >>> > ? ? %%% this next one will return {error, closed} >>> > ? ? {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, 0}, >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >>> > {keyfile, >>> > "controller.key"} ]). >>> > >>> > The python side gets the connection both times. Bizarrely, if I retry a >>> > connection after getting {error, closed} it works. >>> > Any insight? >>> > Dan >>> > >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >> >> > From dgriff1@REDACTED Tue Jul 5 16:15:12 2011 From: dgriff1@REDACTED (Daniel Griffin) Date: Tue, 5 Jul 2011 09:15:12 -0500 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: Thanks, I will do that today and respond with results. Dan On Tue, Jul 5, 2011 at 8:12 AM, Ingela Andin wrote: > Hi! > > I have not actually pinpointed the problem but I got my test-case to fail > on > R14B01 but runs fine on R14B02 and R1B03 and on the main development > branch. > Hence upgrade and let me know how it goes. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/7/4 Ingela Andin : > > Hi again! > > > > 2011/7/4 Daniel Griffin : > >> The twisted python server defaults to accepting SSL 2 or 3. I am using > >> erlang R14B01. Is there some rule saying that you can't have multiple > SSL > >> connections between 2 programs? > > > > No, it should not be a problem. > > > >> What sort of timing issues could I be having? > > > > I will investigate tomorrow if there could be some sort of timing > > issue. I am not saying > > there is, I am just saying your description of what is happening > > makes me think of strange timing issues, (even though I can not think > > of a reason there should be sush an isssue at the moment) but I had > > very little to go on. Knowing which version of the code you are > > running makes it easier. If you have the possibility to try the > > latest version of the code or atleast R14B03 > > that would be good. > > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > >> Thanks, > >> Dan > >> On Mon, Jul 4, 2011 at 4:00 AM, Ingela Andin > wrote: > >>> > >>> Hi! > >>> > >>> It sounds like you are experiencing some kind of timing problem. > >>> I tried to write a test case to repeat it but I am not getting > >>> any problem. Which ssl-version you are running? > >>> > >>> Regards Ingela Erlang/OTP team - Ericsson AB > >>> > >>> 2011/7/2 Daniel Griffin : > >>> > I am using SSL sockets to talk between an erlang program and a > twisted > >>> > python program. Different processes in my erlang app occasionally > have > >>> > to > >>> > open connections and the first one opens fine, then the second errors > >>> > with > >>> > {error, closed}. > >>> > I don't know if I am misunderstanding something in SSL or Erlang. The > >>> > simplest client code I could come up with is: > >>> > run_test()-> > >>> > ssl:start(), > >>> > %%% works fine > >>> > {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, 0}, > >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, > >>> > {keyfile, > >>> > "controller.key"} ]), > >>> > %%% this next one will return {error, closed} > >>> > {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, > 0}, > >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, > >>> > {keyfile, > >>> > "controller.key"} ]). > >>> > > >>> > The python side gets the connection both times. Bizarrely, if I retry > a > >>> > connection after getting {error, closed} it works. > >>> > Any insight? > >>> > Dan > >>> > > >>> > > >>> > _______________________________________________ > >>> > erlang-questions mailing list > >>> > erlang-questions@REDACTED > >>> > http://erlang.org/mailman/listinfo/erlang-questions > >>> > > >>> > > >> > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Tue Jul 5 16:22:54 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 5 Jul 2011 07:22:54 -0700 (PDT) Subject: [erlang-questions] use several log files Message-ID: <1309875774.9776.YahooMailNeo@web112613.mail.gq1.yahoo.com> I'm using mochiweb to serve several virtual hosts. I'm using error_logger:info_msg/2 and error_logger:error_msg/2. Is it possible to have a separate log file for each virtual host? Thanks From drew.varner@REDACTED Tue Jul 5 16:28:42 2011 From: drew.varner@REDACTED (Drew Varner) Date: Tue, 5 Jul 2011 18:28:42 +0400 Subject: [erlang-questions] Event-based ASN.1 Message-ID: <7E887D04-8AA1-43A8-BBA1-94F3875B181A@redops.org> I am interested in a parsing a large ASN.1 structure, a massive CRL. Is there a way to handle it using an event-based technique using Erlang's built-in ASN.1 library? Looking at the generated encode/decode methods, it looks like I have no choice but to bring the entire binary into memory. Is there an event-based piece I am missing or some fun() magic I can use? Thanks, Drew From ericbmerritt@REDACTED Tue Jul 5 17:05:53 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 5 Jul 2011 10:05:53 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: <668896959.52301309823873219.JavaMail.root@zimbra> References: <668896959.52301309823873219.JavaMail.root@zimbra> Message-ID: I understand and that makes sense, but it does raise a potential problem. That is group leaders are created but never destroyed. In the case of the process itself, this is probably ok. The developer will have to create some mechanism to know when his group leader is no longer useful and have it die on its own. However, I didn't see any mechanisms to unregister group leaders and reclaim the memory alloced when they are created. That could be me missing something when I was looking at the code, if so then the point is moot, however if not that's something that would probably need to be added to make this workable. Of course, that still leaves the problem that if a group leader dies unexpectedly, or the user mis-codes his application and forgets to unregister a group leader when it dies, then he has a memory leak. You could say that its a users problem and go from there, but leaving around unreferenced memory just doesn't smell right. (I know, we do that already with the atom table, but it doesn't make it a good thing) On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding wrote: > ----- "Eric Merritt" wrote: > >> > >> > >> >> Just out of curiosity does a group leader go away when a group >> dies? >> > >> > Define group death? >> > >> >> When when all processes in a group have terminated, either normally >> or >> abnormally. > > Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader. > > Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat. > > Robert > From yrashk@REDACTED Tue Jul 5 17:31:36 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Tue, 5 Jul 2011 08:31:36 -0700 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: <668896959.52301309823873219.JavaMail.root@zimbra> Message-ID: Right now there is no way to remove a group leader from process, but it is not inherently impossible to do so, just was out of scope of this prototype. On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt wrote: > I understand and that makes sense, but it does raise a potential > problem. That is group leaders are created but never destroyed. In the > case of the process itself, this is probably ok. The developer will > have to create some mechanism to know when his group leader is no > longer useful and have it die on its own. However, I didn't see any > mechanisms to unregister group leaders and reclaim the memory alloced > when they are created. That could be me missing something when I was > looking at the code, if so then the point is moot, however if not > that's something that would probably need to be added to make this > workable. Of course, that still leaves the problem that if a group > leader dies unexpectedly, or the user mis-codes his application and > forgets to unregister a group leader when it dies, then he has a > memory leak. You could say that its a users problem and go from there, > but leaving around unreferenced memory just doesn't smell right. (I > know, we do that already with the atom table, but it doesn't make it a > good thing) > > On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding > wrote: >> ----- "Eric Merritt" wrote: >> >>> > >>> > >>> >> Just out of curiosity does a group leader go away when a group >>> dies? >>> > >>> > Define group death? >>> > >>> >>> When when all processes in a group have terminated, either normally >>> or >>> abnormally. >> >> Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader. >> >> Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat. >> >> Robert >> > From ericbmerritt@REDACTED Tue Jul 5 17:34:52 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 5 Jul 2011 10:34:52 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: References: <668896959.52301309823873219.JavaMail.root@zimbra> Message-ID: I wouldn't expect in in the prototype. I am more thinking of the end result. On Tue, Jul 5, 2011 at 10:31 AM, Yurii Rashkovskii wrote: > Right now there is no way to remove a group leader from process, but > it is not inherently impossible to do so, just was out of scope of > this prototype. > > On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt wrote: >> I understand and that makes sense, but it does raise a potential >> problem. That is group leaders are created but never destroyed. In the >> case of the process itself, this is probably ok. The developer will >> have to create some mechanism to know when his group leader is no >> longer useful and have it die on its own. However, I didn't see any >> mechanisms to unregister group leaders and reclaim the memory alloced >> when they are created. That could be me missing something when I was >> looking at the code, if so then the point is moot, however if not >> that's something that would probably need to be added to make this >> workable. Of course, that still leaves the problem that if a group >> leader dies unexpectedly, or the user mis-codes his application and >> forgets to unregister a group leader when it dies, then he has a >> memory leak. You could say that its a users problem and go from there, >> but leaving around unreferenced memory just doesn't smell right. (I >> know, we do that already with the atom table, but it doesn't make it a >> good thing) >> >> On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding >> wrote: >>> ----- "Eric Merritt" wrote: >>> >>>> > >>>> > >>>> >> Just out of curiosity does a group leader go away when a group >>>> dies? >>>> > >>>> > Define group death? >>>> > >>>> >>>> When when all processes in a group have terminated, either normally >>>> or >>>> abnormally. >>> >>> Today there is nothing in the system which keeps track of the processes in a group, process groups don't really exist at all. A "process group" is those processes which have the same group leader. The group leader has no information about which processes have it as group leader. You can make any process a group leader by setting it as group leader. >>> >>> Process groups were explicitly added in such a way as NOT to impose any structure or hierarchy on processes, process space is flat. >>> >>> Robert >>> >> > From freza@REDACTED Tue Jul 5 18:07:54 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 5 Jul 2011 17:07:54 +0100 Subject: [erlang-questions] process priority In-Reply-To: References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> <20110704175615.GB16369@hanele> Message-ID: <20110705160754.GA1347@hanele> # Mazen Harake 2011-07-05: > Writing to mnesia/dets requires no locking, nothing mentionable > anyway, since the messages (internal ones) are serialized just like > your gen_server will have its messages serialized (also, use > dirty-operations). I was talking about VM level locks -- but like I said, I don't know if the impact is measurable here. > This creates end to end flow control, actually even more so because > you won't need the extra process (your logging process) in between > the write. By end-to-end I mean a feedback between message producers and their consumer (or consumers) -- I don't see how you get such behaviour with table-based approach. What prevents producers from generating messages faster than consumer/s can read them? > Perhaps your small 100B messages work so well with delayed_write > because you can write many of them to memory before they are flushed > to a file thus not hogging the disk, but I the bigger messages you > have the larger your in memory size needs to be to to avoid this. Sure, delayed_write parameters are configurable in the library I have in mind. It's really more about avoiding OS overhead for many writes, the disk itself just has to be fast enough to handle the load -- if it's not, every buffer wil overrun eventually. It's also a matter of how much delay are you willing to tolerate between enqueing message and seeing it on disk; and how many messages are you willing to lose on tragical VM crash. > Delayed write does of course work well but I have experience that says > that writing and buffering it up in tables can be helpful to avoid > disk thrashing when messages are large (or higher volume). I don't > remember exactly how much throughput we had (and I don't want to guess > since it will be mere speculation without having hard data) but it > helped immensely. > > So I guess OP now have 2 suggestions which of course isn't bad ;) Certainly. :-) > One should also keep in mind though that different situation may have > different needs, would be interesting to see how they would measure > up. Sure -- you can't get persistent queues with gen_server-based approach for instance; it's designed & optimized for relatively small messages arriving at very high rates. If you can recall some details about your workload (average message size, were they iolists/binaries, if iolists how complex were they, how did flusher process work roughly -- this sort of thing) I could probably measure the two approaches in various situations (different message sizes and producer concurrency levels) over the weekend and share the results (but not the code, sorry, proprietary stuff). The overall lesson I've learnt from this is that gen_server calls are dirt-cheap, with a bit of care here and there. BR, -- Jachym From mevans@REDACTED Tue Jul 5 19:18:51 2011 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 5 Jul 2011 13:18:51 -0400 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: References: Message-ID: <1E2B4AB6-1D4E-43CB-9C6D-55C7688B1269@verivue.com> Yes, I was looking for these slides too Sent from my iPhone On Jun 26, 2011, at 2:20 PM, "Paul Fisher" wrote: > Kenneth, > > Can you publish your slides from your Erlang Factory talk on the OTP Roadmap? I apologize if you have already done so, I was looking on the Erlang Factory site and a link was not available. > > > -- > paul > > director, platform services > alertlogic, inc. > 713-484-8383 x2314 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jwatte@REDACTED Tue Jul 5 19:26:33 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 5 Jul 2011 10:26:33 -0700 Subject: [erlang-questions] json in erlang In-Reply-To: References: <425BDED4-8DFD-475A-AD53-97A2F920CB45@hates.ms> <4E0C1C56.6010708@gmail.com> <3E4C95EA-325D-4452-8162-4A8251FEACEC@erlang-solutions.com> Message-ID: +2. I use mochijson2, and it works, but a native version would be nice. Hopefully it uses the same conventions as mochijson2, so switching over is easy :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. 2011/6/30 Tim Watson > 2011/6/30 Fr?d?ric Trottier-H?bert : > > I would definitely voice my support for jsx to become a standard if it > were an option. > > +1. I'm all for NIF based stuff when the speed is necessary, but I > have a suspicion it'll take longer to get a native implementation into > Erlang/OTP-proper and a pure library that is "fast enough" might be a > better/faster option in terms of having something that is "there" when > it's needed. If OTP was going to come with a native code > implementation as a BIF, that would obviously be dead cool though. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Tue Jul 5 23:27:17 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Wed, 6 Jul 2011 00:27:17 +0300 Subject: [erlang-questions] Unit testing code - whence? Message-ID: Hi List, What are the conventions regarding placement of unit test code and invocation? Considerations: I tend to generate a lot of unit tests - 2x..4x SLOC of the tested module's code is not uncommon. Should the unit test functions be placed in their own module? If so - what is the naming convention for the new modue: 'test_mymod' / 'mymod_test' or something else? What is the preferred way, if there are many small UT functions: function *_test() per UT, or separate into test sets by group, e.g.: api_unit_test_(), internal_unit_test_(). Cheers, Alex. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Jul 5 23:34:37 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 5 Jul 2011 14:34:37 -0700 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: On Tue, Jul 5, 2011 at 2:27 PM, Alex Arnon wrote: > > What are the conventions regarding placement of unit test code and > invocation? > Considerations: > I tend to generate a lot of unit tests - 2x..4x SLOC of the tested module's > code is not uncommon. > Should the unit test functions be placed in their own module? > If so - what is the naming convention for the new modue: 'test_mymod' / > 'mymod_test' or something else? Yes. The convention is to place them in test/mymod_tests.erl. These will get picked up by rebar eunit. > What is the preferred way, if there are many small UT functions: function > *_test() per UT, or separate into test sets by group, e.g.: > api_unit_test_(), internal_unit_test_(). I don't think there is as much consensus for this. I do whatever makes sense at the time. I don't try and group tests together unless it's easier to write the code that way (e.g. with a list comprehension, or if they share the setup/teardown in a foreach). -bob From alex.arnon@REDACTED Tue Jul 5 23:47:51 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Wed, 6 Jul 2011 00:47:51 +0300 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: Thank you! On Wed, Jul 6, 2011 at 12:34 AM, Bob Ippolito wrote: > On Tue, Jul 5, 2011 at 2:27 PM, Alex Arnon wrote: > > > > What are the conventions regarding placement of unit test code and > > invocation? > > Considerations: > > I tend to generate a lot of unit tests - 2x..4x SLOC of the tested > module's > > code is not uncommon. > > Should the unit test functions be placed in their own module? > > If so - what is the naming convention for the new modue: 'test_mymod' / > > 'mymod_test' or something else? > > Yes. The convention is to place them in test/mymod_tests.erl. These > will get picked up by rebar eunit. > > > What is the preferred way, if there are many small UT functions: function > > *_test() per UT, or separate into test sets by group, e.g.: > > api_unit_test_(), internal_unit_test_(). > > I don't think there is as much consensus for this. I do whatever makes > sense at the time. I don't try and group tests together unless it's > easier to write the code that way (e.g. with a list comprehension, or > if they share the setup/teardown in a foreach). > > -bob > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Jul 6 01:04:02 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 5 Jul 2011 23:04:02 +0000 (GMT) Subject: [erlang-questions] Scoped group leaders In-Reply-To: Message-ID: <755748402.58461309907042414.JavaMail.root@zimbra> Conceptually a process is always the member of a group so the very idea of removing a group leader from a process does not make sense. All you can do is assign it another group leader. Robert ----- "Eric Merritt" wrote: > I wouldn't expect in in the prototype. I am more thinking of the end > result. > > On Tue, Jul 5, 2011 at 10:31 AM, Yurii Rashkovskii > wrote: > > Right now there is no way to remove a group leader from process, > but > > it is not inherently impossible to do so, just was out of scope of > > this prototype. > > > > On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt > wrote: > >> I understand and that makes sense, but it does raise a potential > >> problem. That is group leaders are created but never destroyed. In > the > >> case of the process itself, this is probably ok. The developer > will > >> have to create some mechanism to know when his group leader is no > >> longer useful and have it die on its own. However, I didn't see > any > >> mechanisms to unregister group leaders and reclaim the memory > alloced > >> when they are created. That could be me missing something when I > was > >> looking at the code, if so then the point is moot, however if not > >> that's something that would probably need to be added to make this > >> workable. Of course, that still leaves the problem that if a group > >> leader dies unexpectedly, or the user mis-codes his application > and > >> forgets to unregister a group leader when it dies, then he has a > >> memory leak. You could say that its a users problem and go from > there, > >> but leaving around unreferenced memory just doesn't smell right. > (I > >> know, we do that already with the atom table, but it doesn't make > it a > >> good thing) > >> > >> On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding > >> wrote: > >>> ----- "Eric Merritt" wrote: > >>> > >>>> > > >>>> > > >>>> >> Just out of curiosity does a group leader go away when a > group > >>>> dies? > >>>> > > >>>> > Define group death? > >>>> > > >>>> > >>>> When when all processes in a group have terminated, either > normally > >>>> or > >>>> abnormally. > >>> > >>> Today there is nothing in the system which keeps track of the > processes in a group, process groups don't really exist at all. A > "process group" is those processes which have the same group leader. > The group leader has no information about which processes have it as > group leader. You can make any process a group leader by setting it as > group leader. > >>> > >>> Process groups were explicitly added in such a way as NOT to > impose any structure or hierarchy on processes, process space is > flat. > >>> > >>> Robert > >>> > >> > > From robert.virding@REDACTED Wed Jul 6 01:11:19 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 5 Jul 2011 23:11:19 +0000 (GMT) Subject: [erlang-questions] function badarg In-Reply-To: Message-ID: <1176988592.58491309907479523.JavaMail.root@zimbra> For the moment, no. It is said that in R15 errors will not just include in which function they occur but also on which line. You get a badarg error when you make a system call or call to OTP libraries and the arguments are wrong, wrong type or illegal value. Robert ----- "Alexander Kuleshov" wrote: > Hello, > > I have loop/2 function. This is main cycle of processing input data. > > Approximate form of this function: > > loop(Session, State) -> > {1} -> > ..... > loop(Session, State); > {2} -> > ..... > loop(Session, State); > {3} -> > ..... > loop(Session, State); > _ -> > loop(Session, State). > > In fact, this function big. Something about 1000 string of code. > > Sometimes this function crash with: Error in process <0.384.0> on > node > 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} > > Is there instruments or methods to know where it occurs and how it > can > catch this? > > Thank you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ericbmerritt@REDACTED Wed Jul 6 01:14:08 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 5 Jul 2011 18:14:08 -0500 Subject: [erlang-questions] Scoped group leaders In-Reply-To: <755748402.58461309907042414.JavaMail.root@zimbra> References: <755748402.58461309907042414.JavaMail.root@zimbra> Message-ID: I am talking about the situation where no process that where once part of the group exist any longer. When a group has no processes, the group no longer exists, however the memory alloced by this will continue to exist, thats the problem. On Tue, Jul 5, 2011 at 6:04 PM, Robert Virding wrote: > Conceptually a process is always the member of a group so the very idea of removing a group leader from a process does not make sense. All you can do is assign it another group leader. > > Robert > > ----- "Eric Merritt" wrote: > >> I wouldn't expect in in the prototype. I am more thinking of the end >> result. >> >> On Tue, Jul 5, 2011 at 10:31 AM, Yurii Rashkovskii >> wrote: >> > Right now there is no way to remove a group leader from process, >> but >> > it is not inherently impossible to do so, just was out of scope of >> > this prototype. >> > >> > On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt >> wrote: >> >> I understand and that makes sense, but it does raise a potential >> >> problem. That is group leaders are created but never destroyed. In >> the >> >> case of the process itself, this is probably ok. The developer >> will >> >> have to create some mechanism to know when his group leader is no >> >> longer useful and have it die on its own. However, I didn't see >> any >> >> mechanisms to unregister group leaders and reclaim the memory >> alloced >> >> when they are created. That could be me missing something when I >> was >> >> looking at the code, if so then the point is moot, however if not >> >> that's something that would probably need to be added to make this >> >> workable. Of course, that still leaves the problem that if a group >> >> leader dies unexpectedly, or the user mis-codes his application >> and >> >> forgets to unregister a group leader when it dies, then he has a >> >> memory leak. You could say that its a users problem and go from >> there, >> >> but leaving around unreferenced memory just doesn't smell right. >> (I >> >> know, we do that already with the atom table, but it doesn't make >> it a >> >> good thing) >> >> >> >> On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding >> >> wrote: >> >>> ----- "Eric Merritt" wrote: >> >>> >> >>>> > >> >>>> > >> >>>> >> Just out of curiosity does a group leader go away when a >> group >> >>>> dies? >> >>>> > >> >>>> > Define group death? >> >>>> > >> >>>> >> >>>> When when all processes in a group have terminated, either >> normally >> >>>> or >> >>>> abnormally. >> >>> >> >>> Today there is nothing in the system which keeps track of the >> processes in a group, process groups don't really exist at all. A >> "process group" is those processes which have the same group leader. >> The group leader has no information about which processes have it as >> group leader. You can make any process a group leader by setting it as >> group leader. >> >>> >> >>> Process groups were explicitly added in such a way as NOT to >> impose any structure or hierarchy on processes, process space is >> flat. >> >>> >> >>> Robert >> >>> >> >> >> > > From moxford@REDACTED Wed Jul 6 02:09:59 2011 From: moxford@REDACTED (Mike Oxford) Date: Tue, 5 Jul 2011 17:09:59 -0700 Subject: [erlang-questions] WTF? Message-ID: Application tree was working...has been for weeks. Ran rebar and it crashed, shit all over my nodes (again.) No problem, been there done that, fix it up... Now when I run it (after rebar's generate) it gives me the "undef start/2" error. Spend hours trying to figure out everything from how my lib_dirs got changed, env, everything, full cleans, everything. Pull a backup ... it works. Hand merge all new changes... works, runs fine. Nuke the directory and copy the backup into the directory....undef start/2. Copy the entire tree to another directory ... runs fine. What/where/WTF has been corrupted that running it from a specific directory would cause issues? /opt/name/proj <-- undef start/2, used to work /opt/name/wtf/proj <-- works My lib_dirs have not changed, binary cp -r between the two directories so they're identical. Rebar may have caused it, but now that it's in this completely f'd up state I need to figure out how to get it OUT of this state, since rebar's just a packaging system in front of erlexec. So ... two identical directory structures, one works and one does not. Again, it's been working for weeks .. this is not a fundamental issue but more "state." I have -not- rebooted. I want to understand this fully before I push in into production. Anyone know of a good place to start digging? TIA. -mox From dgriff1@REDACTED Wed Jul 6 02:27:03 2011 From: dgriff1@REDACTED (Daniel Griffin) Date: Tue, 5 Jul 2011 19:27:03 -0500 Subject: [erlang-questions] SSL {error, closed} In-Reply-To: References: Message-ID: Upgrading fixed it. Thanks for the help, Dan On Tue, Jul 5, 2011 at 9:15 AM, Daniel Griffin wrote: > Thanks, I will do that today and respond with results. > > Dan > > > On Tue, Jul 5, 2011 at 8:12 AM, Ingela Andin wrote: > >> Hi! >> >> I have not actually pinpointed the problem but I got my test-case to fail >> on >> R14B01 but runs fine on R14B02 and R1B03 and on the main development >> branch. >> Hence upgrade and let me know how it goes. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/7/4 Ingela Andin : >> > Hi again! >> > >> > 2011/7/4 Daniel Griffin : >> >> The twisted python server defaults to accepting SSL 2 or 3. I am using >> >> erlang R14B01. Is there some rule saying that you can't have multiple >> SSL >> >> connections between 2 programs? >> > >> > No, it should not be a problem. >> > >> >> What sort of timing issues could I be having? >> > >> > I will investigate tomorrow if there could be some sort of timing >> > issue. I am not saying >> > there is, I am just saying your description of what is happening >> > makes me think of strange timing issues, (even though I can not think >> > of a reason there should be sush an isssue at the moment) but I had >> > very little to go on. Knowing which version of the code you are >> > running makes it easier. If you have the possibility to try the >> > latest version of the code or atleast R14B03 >> > that would be good. >> > >> > Regards Ingela Erlang/OTP team - Ericsson AB >> > >> >> Thanks, >> >> Dan >> >> On Mon, Jul 4, 2011 at 4:00 AM, Ingela Andin >> wrote: >> >>> >> >>> Hi! >> >>> >> >>> It sounds like you are experiencing some kind of timing problem. >> >>> I tried to write a test case to repeat it but I am not getting >> >>> any problem. Which ssl-version you are running? >> >>> >> >>> Regards Ingela Erlang/OTP team - Ericsson AB >> >>> >> >>> 2011/7/2 Daniel Griffin : >> >>> > I am using SSL sockets to talk between an erlang program and a >> twisted >> >>> > python program. Different processes in my erlang app occasionally >> have >> >>> > to >> >>> > open connections and the first one opens fine, then the second >> errors >> >>> > with >> >>> > {error, closed}. >> >>> > I don't know if I am misunderstanding something in SSL or Erlang. >> The >> >>> > simplest client code I could come up with is: >> >>> > run_test()-> >> >>> > ssl:start(), >> >>> > %%% works fine >> >>> > {ok, Conn} = ssl:connect("localhost", 2000, [binary, {packet, >> 0}, >> >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >> >>> > {keyfile, >> >>> > "controller.key"} ]), >> >>> > %%% this next one will return {error, closed} >> >>> > {ok, Conn1} = ssl:connect("localhost", 2000, [binary, {packet, >> 0}, >> >>> > {keepalive, true}, {active, false}, { certfile, "controller.crt"}, >> >>> > {keyfile, >> >>> > "controller.key"} ]). >> >>> > >> >>> > The python side gets the connection both times. Bizarrely, if I >> retry a >> >>> > connection after getting {error, closed} it works. >> >>> > Any insight? >> >>> > Dan >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > erlang-questions mailing list >> >>> > erlang-questions@REDACTED >> >>> > http://erlang.org/mailman/listinfo/erlang-questions >> >>> > >> >>> > >> >> >> >> >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Jul 6 02:34:23 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 6 Jul 2011 00:34:23 +0000 (GMT) Subject: [erlang-questions] Scoped group leaders In-Reply-To: Message-ID: <1285524784.58801309912463694.JavaMail.root@zimbra> It's not that bad, the only memory allocated is for the group leader process. ----- "Eric Merritt" wrote: > I am talking about the situation where no process that where once > part > of the group exist any longer. When a group has no processes, the > group no longer exists, however the memory alloced by this will > continue to exist, thats the problem. > > On Tue, Jul 5, 2011 at 6:04 PM, Robert Virding > wrote: > > Conceptually a process is always the member of a group so the very > idea of removing a group leader from a process does not make sense. > All you can do is assign it another group leader. > > > > Robert > > > > ----- "Eric Merritt" wrote: > > > >> I wouldn't expect in in the prototype. I am more thinking of the > end > >> result. > >> > >> On Tue, Jul 5, 2011 at 10:31 AM, Yurii Rashkovskii > > >> wrote: > >> > Right now there is no way to remove a group leader from process, > >> but > >> > it is not inherently impossible to do so, just was out of scope > of > >> > this prototype. > >> > > >> > On Tue, Jul 5, 2011 at 8:05 AM, Eric Merritt > >> wrote: > >> >> I understand and that makes sense, but it does raise a > potential > >> >> problem. That is group leaders are created but never destroyed. > In > >> the > >> >> case of the process itself, this is probably ok. The developer > >> will > >> >> have to create some mechanism to know when his group leader is > no > >> >> longer useful and have it die on its own. However, I didn't see > >> any > >> >> mechanisms to unregister group leaders and reclaim the memory > >> alloced > >> >> when they are created. That could be me missing something when > I > >> was > >> >> looking at the code, if so then the point is moot, however if > not > >> >> that's something that would probably need to be added to make > this > >> >> workable. Of course, that still leaves the problem that if a > group > >> >> leader dies unexpectedly, or the user mis-codes his application > >> and > >> >> forgets to unregister a group leader when it dies, then he has > a > >> >> memory leak. You could say that its a users problem and go from > >> there, > >> >> but leaving around unreferenced memory just doesn't smell > right. > >> (I > >> >> know, we do that already with the atom table, but it doesn't > make > >> it a > >> >> good thing) > >> >> > >> >> On Mon, Jul 4, 2011 at 6:57 PM, Robert Virding > >> >> wrote: > >> >>> ----- "Eric Merritt" wrote: > >> >>> > >> >>>> > > >> >>>> > > >> >>>> >> Just out of curiosity does a group leader go away when a > >> group > >> >>>> dies? > >> >>>> > > >> >>>> > Define group death? > >> >>>> > > >> >>>> > >> >>>> When when all processes in a group have terminated, either > >> normally > >> >>>> or > >> >>>> abnormally. > >> >>> > >> >>> Today there is nothing in the system which keeps track of the > >> processes in a group, process groups don't really exist at all. A > >> "process group" is those processes which have the same group > leader. > >> The group leader has no information about which processes have it > as > >> group leader. You can make any process a group leader by setting it > as > >> group leader. > >> >>> > >> >>> Process groups were explicitly added in such a way as NOT to > >> impose any structure or hierarchy on processes, process space is > >> flat. > >> >>> > >> >>> Robert > >> >>> > >> >> > >> > > > From moxford@REDACTED Wed Jul 6 02:52:34 2011 From: moxford@REDACTED (Mike Oxford) Date: Tue, 5 Jul 2011 17:52:34 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: erl running inside of a backgrounded emacs in the original directory seems to have been the culprit. Killed it off, nuked the directory, used the merged backup with the rebuilt-nodes and things are good again. If two things are going to kill Erlang's uptake it's not going to be the syntax, the performance, the functional-way-of-thinking... it'll be the terrible error messages and the arcane build/packaging, both of which make Erlang feel like a house of cards you can't even look at wrong and, thus, give no confidence in actually running it in production. (Not to mention just plain annoying devs to the point they walk away from it, which is what kills more languages over time than anything else.) Party on... -mox On Tue, Jul 5, 2011 at 5:09 PM, Mike Oxford wrote: > Application tree was working...has been for weeks. > > Ran rebar and it crashed, shit all over my nodes (again.) ?No problem, > been there done that, fix it up... > > Now when I run it (after rebar's generate) it gives me the "undef > start/2" error. > > Spend hours trying to figure out everything from how my lib_dirs got > changed, env, everything, full cleans, everything. > > Pull a backup ... it works. > Hand merge all new changes... works, runs fine. > Nuke the directory and copy the backup into the directory....undef start/2. > Copy the entire tree to another directory ... runs fine. > > What/where/WTF has been corrupted that running it from a specific > directory would cause issues? > > /opt/name/proj <-- undef start/2, used to work > /opt/name/wtf/proj <-- works > > My lib_dirs have not changed, binary cp -r between the two directories > so they're identical. > > Rebar may have caused it, but now that it's in this completely f'd up > state I need to figure out how to get it OUT of this state, since > rebar's just a packaging system in front of erlexec. > > So ... two identical directory structures, one works and one does not. > > Again, it's been working for weeks .. this is not a fundamental issue > but more "state." > > I have -not- rebooted. ?I want to understand this fully before I push > in into production. > > Anyone know of a good place to start digging? > > TIA. > > -mox > From kennethstone@REDACTED Wed Jul 6 03:04:20 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Tue, 5 Jul 2011 20:04:20 -0500 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: I love erlang's syntax <3 rebar can be a big magical and scary... On Tue, Jul 5, 2011 at 7:52 PM, Mike Oxford wrote: > erl running inside of a backgrounded emacs in the original directory > seems to have been the culprit. > Killed it off, nuked the directory, used the merged backup with the > rebuilt-nodes and things are good again. > > If two things are going to kill Erlang's uptake it's not going to be > the syntax, the performance, the functional-way-of-thinking... it'll > be the terrible error messages and the arcane build/packaging, both of > which make Erlang feel like a house of cards you can't even look at > wrong and, thus, give no confidence in actually running it in > production. (Not to mention just plain annoying devs to the point > they walk away from it, which is what kills more languages over time > than anything else.) > > Party on... > > -mox > > > On Tue, Jul 5, 2011 at 5:09 PM, Mike Oxford wrote: > > Application tree was working...has been for weeks. > > > > Ran rebar and it crashed, shit all over my nodes (again.) No problem, > > been there done that, fix it up... > > > > Now when I run it (after rebar's generate) it gives me the "undef > > start/2" error. > > > > Spend hours trying to figure out everything from how my lib_dirs got > > changed, env, everything, full cleans, everything. > > > > Pull a backup ... it works. > > Hand merge all new changes... works, runs fine. > > Nuke the directory and copy the backup into the directory....undef > start/2. > > Copy the entire tree to another directory ... runs fine. > > > > What/where/WTF has been corrupted that running it from a specific > > directory would cause issues? > > > > /opt/name/proj <-- undef start/2, used to work > > /opt/name/wtf/proj <-- works > > > > My lib_dirs have not changed, binary cp -r between the two directories > > so they're identical. > > > > Rebar may have caused it, but now that it's in this completely f'd up > > state I need to figure out how to get it OUT of this state, since > > rebar's just a packaging system in front of erlexec. > > > > So ... two identical directory structures, one works and one does not. > > > > Again, it's been working for weeks .. this is not a fundamental issue > > but more "state." > > > > I have -not- rebooted. I want to understand this fully before I push > > in into production. > > > > Anyone know of a good place to start digging? > > > > TIA. > > > > -mox > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 6 07:33:34 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 5 Jul 2011 22:33:34 -0700 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: Unit tests in a different module means you have to open your module up a lot more than if the tests are in the same module. Generally, I write the code, then I include_lib eunit.hrl, then I write the unit tests. That can all be wrapped in conditional compilation if you need to. However, when just developing new functions, I typically put the tests right by the functions they're testing, and only when I'm done do I move them to the end of the file. Works for me/us! Your mileage may vary. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 5, 2011 at 2:27 PM, Alex Arnon wrote: > Hi List, > > What are the conventions regarding placement of unit test code and > invocation? > Considerations: > I tend to generate a lot of unit tests - 2x..4x SLOC of the tested module's > code is not uncommon. > Should the unit test functions be placed in their own module? > If so - what is the naming convention for the new modue: 'test_mymod' / > 'mymod_test' or something else? > What is the preferred way, if there are many small UT functions: function > *_test() per UT, or separate into test sets by group, e.g.: > api_unit_test_(), internal_unit_test_(). > > Cheers, > Alex. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 6 07:38:51 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 5 Jul 2011 22:38:51 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: Why re-invent make and make it worse? No, really. Typically, you'll want to use code generation, and multiple languages, and perhaps custom build steps, unit tests, functional tests, ... All of which works great in Make with some smart wildcards and implicit rules. No offense to the rebar lovers, but you can pry GNU make out of my cold, dead hands; not before :-) Our Erlang build uses a Python-based unit test driver, builds a few C-based tools, and generates Erlang, Python and PHP marshalling glue for our network protocol. Oh, and also acceptance tests our bash-based deployment scripts. Rebar didn't deal with that when I tried. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 5, 2011 at 6:04 PM, Kenny Stone wrote: > I love erlang's syntax <3 > > rebar can be a big magical and scary... > > > On Tue, Jul 5, 2011 at 7:52 PM, Mike Oxford wrote: > >> erl running inside of a backgrounded emacs in the original directory >> seems to have been the culprit. >> Killed it off, nuked the directory, used the merged backup with the >> rebuilt-nodes and things are good again. >> >> If two things are going to kill Erlang's uptake it's not going to be >> the syntax, the performance, the functional-way-of-thinking... it'll >> be the terrible error messages and the arcane build/packaging, both of >> which make Erlang feel like a house of cards you can't even look at >> wrong and, thus, give no confidence in actually running it in >> production. (Not to mention just plain annoying devs to the point >> they walk away from it, which is what kills more languages over time >> than anything else.) >> >> Party on... >> >> -mox >> >> >> On Tue, Jul 5, 2011 at 5:09 PM, Mike Oxford wrote: >> > Application tree was working...has been for weeks. >> > >> > Ran rebar and it crashed, shit all over my nodes (again.) No problem, >> > been there done that, fix it up... >> > >> > Now when I run it (after rebar's generate) it gives me the "undef >> > start/2" error. >> > >> > Spend hours trying to figure out everything from how my lib_dirs got >> > changed, env, everything, full cleans, everything. >> > >> > Pull a backup ... it works. >> > Hand merge all new changes... works, runs fine. >> > Nuke the directory and copy the backup into the directory....undef >> start/2. >> > Copy the entire tree to another directory ... runs fine. >> > >> > What/where/WTF has been corrupted that running it from a specific >> > directory would cause issues? >> > >> > /opt/name/proj <-- undef start/2, used to work >> > /opt/name/wtf/proj <-- works >> > >> > My lib_dirs have not changed, binary cp -r between the two directories >> > so they're identical. >> > >> > Rebar may have caused it, but now that it's in this completely f'd up >> > state I need to figure out how to get it OUT of this state, since >> > rebar's just a packaging system in front of erlexec. >> > >> > So ... two identical directory structures, one works and one does not. >> > >> > Again, it's been working for weeks .. this is not a fundamental issue >> > but more "state." >> > >> > I have -not- rebooted. I want to understand this fully before I push >> > in into production. >> > >> > Anyone know of a good place to start digging? >> > >> > TIA. >> > >> > -mox >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bpdp.erlang@REDACTED Wed Jul 6 07:54:16 2011 From: bpdp.erlang@REDACTED (Bambang Purnomosidi D. P.) Date: Wed, 6 Jul 2011 12:54:16 +0700 Subject: [erlang-questions] agner install -> fail, AGNER_INSTALL_PREFIX problem in Linux Message-ID: Hi, I use agner to install a package and encounter a problem in installing process: [Installing...] FAILURE: {'EXIT',<0.67.0>, {{badmatch,{error,enoent}}, [{agner_fetch,install_dirs,1}, {agner_fetch,installable,2}, {gen_fsm2,handle_msg,7}, {proc_lib,init_p_do_apply,3}]}} Reading the error message, I guess that it was because the installation dir is not installable. This can be accepted since I ran agner not as root or using sudo. I've read the agner wiki and it says that we can use AGNER_INSTALL_PREFIX to define installation dir but the error is still the same. Am I missing something? Any hint(s)? Thank you :) -- bpdp -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianfranco.alongi@REDACTED Wed Jul 6 07:56:28 2011 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Wed, 6 Jul 2011 06:56:28 +0100 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: I disagree. Putting your tests in another module will actually force you to think more of your design, making the application/lib more testable without being necessarily more "open". Another huge win is that you will have a cleaner implementation module. And you will be able to easily mix and match revisions of code and tests. /G On 6 Jul 2011 06:33, "Jon Watte" wrote: Unit tests in a different module means you have to open your module up a lot more than if the tests are in the same module. Generally, I write the code, then I include_lib eunit.hrl, then I write the unit tests. That can all be wrapped in conditional compilation if you need to. However, when just developing new functions, I typically put the tests right by the functions they're testing, and only when I'm done do I move them to the end of the file. Works for me/us! Your mileage may vary. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 5, 2011 at 2:27 PM, Alex Arnon wrote: > > > > Hi List, > > > > What are the conventions regarding placement of unit test code and > invocation? > > Co... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Wed Jul 6 08:20:41 2011 From: v@REDACTED (Valentin Micic) Date: Wed, 6 Jul 2011 08:20:41 +0200 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: Are you blaming a language for the application's misbehavior? Blaming a runtime environment for not producing application specific errors? Really? V/ On 06 Jul 2011, at 2:52 AM, Mike Oxford wrote: > erl running inside of a backgrounded emacs in the original directory > seems to have been the culprit. > Killed it off, nuked the directory, used the merged backup with the > rebuilt-nodes and things are good again. > > If two things are going to kill Erlang's uptake it's not going to be > the syntax, the performance, the functional-way-of-thinking... it'll > be the terrible error messages and the arcane build/packaging, both of > which make Erlang feel like a house of cards you can't even look at > wrong and, thus, give no confidence in actually running it in > production. (Not to mention just plain annoying devs to the point > they walk away from it, which is what kills more languages over time > than anything else.) > > Party on... > > -mox > > > On Tue, Jul 5, 2011 at 5:09 PM, Mike Oxford wrote: >> Application tree was working...has been for weeks. >> >> Ran rebar and it crashed, shit all over my nodes (again.) No problem, >> been there done that, fix it up... >> >> Now when I run it (after rebar's generate) it gives me the "undef >> start/2" error. >> >> Spend hours trying to figure out everything from how my lib_dirs got >> changed, env, everything, full cleans, everything. >> >> Pull a backup ... it works. >> Hand merge all new changes... works, runs fine. >> Nuke the directory and copy the backup into the directory....undef start/2. >> Copy the entire tree to another directory ... runs fine. >> >> What/where/WTF has been corrupted that running it from a specific >> directory would cause issues? >> >> /opt/name/proj <-- undef start/2, used to work >> /opt/name/wtf/proj <-- works >> >> My lib_dirs have not changed, binary cp -r between the two directories >> so they're identical. >> >> Rebar may have caused it, but now that it's in this completely f'd up >> state I need to figure out how to get it OUT of this state, since >> rebar's just a packaging system in front of erlexec. >> >> So ... two identical directory structures, one works and one does not. >> >> Again, it's been working for weeks .. this is not a fundamental issue >> but more "state." >> >> I have -not- rebooted. I want to understand this fully before I push >> in into production. >> >> Anyone know of a good place to start digging? >> >> TIA. >> >> -mox >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Jul 6 09:59:26 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 06 Jul 2011 09:59:26 +0200 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: <4E1415DE.9040908@dev-extend.eu> On 07/06/2011 07:38 AM, Jon Watte wrote: > Why re-invent make and make it worse? No, really. Typically, you'll want > to use code generation, and multiple languages, and perhaps custom build > steps, unit tests, functional tests, ... All of which works great in > Make with some smart wildcards and implicit rules. No offense to the > rebar lovers, but you can pry GNU make out of my cold, dead hands; not > before :-) > > Our Erlang build uses a Python-based unit test driver, builds a few > C-based tools, and generates Erlang, Python and PHP marshalling glue for > our network protocol. Oh, and also acceptance tests our bash-based > deployment scripts. > Rebar didn't deal with that when I tried. rebar isn't trying to replace make. Most people call rebar from inside a Makefile. -- Lo?c Hoguin Dev:Extend From erlang@REDACTED Wed Jul 6 10:48:04 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Jul 2011 10:48:04 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 5:11 PM, Zabrane Mickael wrote: > Hi guys, > I'd like your help for this small Web servers challenge. > The idea is ver simple: measure "how fast" you can serve a "100 bytes" file > (attached as "small.html") Why measure this? Serving static content is one of the least interesting things I can think of to study. Any network worth the name won't bother to access the "root" server but will cache the content upstream. How fast you can serve up dynamic content is far more interesting, since it can't be cached. Something that evokes a computation on the server is far more interesting, and something that stresses the concurrency. How about processing send a GET /fac/N command (ie return factorial N) The server should 1) send factorial N 2) wait 30 seconds 3) send factorial N (again) 4) close the socket The 30 seconds gap will cause lots of nastiness in the server. Such a benchmark tests many useful things (for example the inter-session protection mechanism, as soon as we perform computation on behalf of the client we have to protect server sessions against each other, secondly handling concurrency. If we wait 30 sections and do 10K requests/sec, we going to build up 300K processes, which tests a lot of internal stuff) /Joe > in "less than 01 second"?by using one of the?following web servers: > - Yaws (http://yaws.hyber.org/download/yaws-1.90.tar.gz) > - Mochiweb (https://github.com/mochi/mochiweb) > - Misultin (https://github.com/ostinelli/misultin) > - Cowboy (https://github.com/extend/cowboy) > - Bare metal web server using "gen_tcp" > The rules are very simple : > 1. The first time you served the file, you MUST read it from disk. > All subsequent requests to that file can use any caching mechanism! > 2. Any (hidden) trick is allowed (eg. prim_inet, SMP enabled ...) > 3. Erlang versions?R14B to R14B03 > 4. The HTTP client to use for this benchmark needs to be "Apache AB" > (http://httpd.apache.org/docs/2.0/programs/ab.html)?with: > $ ab -n 200000 -c 100 -t1 -k "http://127.0.0.1:8080/small.html" > I hope that the authors if these web servers will also contribute to this > bench as they > know their software better than anybody else. > Challenge's open ;-) > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From gleber.p@REDACTED Wed Jul 6 11:04:46 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 6 Jul 2011 11:04:46 +0200 Subject: [erlang-questions] function badarg In-Reply-To: <1176988592.58491309907479523.JavaMail.root@zimbra> References: <1176988592.58491309907479523.JavaMail.root@zimbra> Message-ID: try using dbg module to set up tracing of received messages. This will at least tell you which branch might have matched and executed. On Wed, Jul 6, 2011 at 01:11, Robert Virding wrote: > For the moment, no. It is said that in R15 errors will not just include in which function they occur but also on which line. You get a badarg error when you make a system call or call to OTP libraries and the arguments are wrong, wrong type or illegal value. > > Robert > > ----- "Alexander Kuleshov" wrote: > >> Hello, >> >> I have loop/2 function. This is main cycle of processing input data. >> >> Approximate form of this function: >> >> loop(Session, State) -> >> ? {1} -> >> ? ? ? ? ..... >> ? ? ? ?loop(Session, State); >> {2} -> >> ? ? ? ? ..... >> ? ? ? ?loop(Session, State); >> {3} -> >> ? ? ? ? ..... >> ? ? ? ?loop(Session, State); >> _ -> >> ? ? ?loop(Session, State). >> >> In fact, this function big. Something about 1000 string of code. >> >> Sometimes this function crash with: Error in process <0.384.0> on >> node >> 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} >> >> Is there instruments or methods to know where it occurs and how it >> can >> catch this? >> >> Thank you. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mazen.harake@REDACTED Wed Jul 6 11:30:31 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Wed, 6 Jul 2011 11:30:31 +0200 Subject: [erlang-questions] process priority In-Reply-To: <20110705160754.GA1347@hanele> References: <4E11968E.6010402@gmail.com> <70D6C4BB-4C0F-4AEE-AE0E-78358B04EF77@erlang-solutions.com> <4E11A77A.3040504@gmail.com> <20110704175615.GB16369@hanele> <20110705160754.GA1347@hanele> Message-ID: On 5 July 2011 18:07, Jachym Holecek wrote: > # Mazen Harake 2011-07-05: >> Writing to mnesia/dets requires no locking, nothing mentionable >> anyway, since the messages (internal ones) are serialized just like >> your gen_server will have its messages serialized (also, use >> dirty-operations). > > I was talking about VM level locks -- but like I said, I don't > know if the impact is measurable here. > Not sure what you mean by this? The two scenarios (whether a logging goes to a gen_server or to a table process) are the same. I don't think locks are measurable in this instance. > >> This creates end to end flow control, actually even more so because >> you won't need the extra process (your logging process) in between >> the write. > > By end-to-end I mean a feedback between message producers and their > consumer (or consumers) -- I don't see how you get such behaviour > with table-based approach. What prevents producers from generating > messages faster than consumer/s can read them? > My mistake, I misunderstood your use of flow control which translated wrongly in my mind :). The inherited flow control is: the more processes try to write to a log table, the slower it will get BUT it will only be slower, not stopped (due to io hogging). That is the idea with using a table. >> Perhaps your small 100B messages work so well with delayed_write >> because you can write many of them to memory before they are flushed >> to a file thus not hogging the disk, but I the bigger messages you >> have the larger your in memory size needs to be to to avoid this. > > Sure, delayed_write parameters are configurable in the library I have > in mind. It's really more about avoiding OS overhead for many writes, > the disk itself just has to be fast enough to handle the load -- if > it's not, every buffer wil overrun eventually. It's also a matter of > how much delay are you willing to tolerate between enqueing message > and seeing it on disk; and how many messages are you willing to lose > on tragical VM crash. > The point is to use the disk as little as possible. If you have a fast disk it will perform better over all, of course, but as you said the OS overhead for many writes is what you try to avoid. So buffering up and writing everything is the better way to do it. Now if this is by using delayed_write or by buffering in tables is another question. At least we agree on this :) >> Delayed write does of course work well but I have experience that says >> that writing and buffering it up in tables can be helpful to avoid >> disk thrashing when messages are large (or higher volume). I don't >> remember exactly how much throughput we had (and I don't want to guess >> since it will be mere speculation without having hard data) but it >> helped immensely. >> >> So I guess OP now have 2 suggestions which of course isn't bad ;) > > Certainly. :-) > >> One should also keep in mind though that different situation may have >> different needs, would be interesting to see how they would measure >> up. > > Sure -- you can't get persistent queues with gen_server-based approach > for instance; it's designed & optimized for relatively small messages > arriving at very high rates. > This makes me curious if you thought I mean that the tables I propose will be disk copies. My suggestion was in memory tables only but having them disk based will make it slower but persistent as you say. > If you can recall some details about your workload (average message size, > were they iolists/binaries, if iolists how complex were they, how did > flusher process work roughly -- this sort of thing) I could probably > measure the two approaches in various situations (different message > sizes and producer concurrency levels) over the weekend and share the > results (but not the code, sorry, proprietary stuff). IIRC. Message size: around 1K +- 200B, iolists maybe (not fact) Messages per second: Don't remember... dare I guess around 10k? (Don't ask me to bet my money on it ;)) Flusher: Just like delayed write but using the table as the buffer. I.e. read size every x second(s), if value > N -> flush(value) else if x' seconds have passed -> flush(value). immediately check for size again. Would be interesting to see how it performs. /M From zabrane3@REDACTED Wed Jul 6 11:45:50 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 6 Jul 2011 11:45:50 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: > Why measure this? Serving static content is one of the least > interesting things I can think of > to study. Any network worth the name won't bother to access the "root" > server but will cache the content upstream. I disagree. That's not what the challenge is about. Why don't just try it and avoid (useless) questions! > How fast you can serve up dynamic content is far more interesting, > since it can't be cached. This will be part of my second challenge of course. For now, let's stick with the first challenge rules please. Regards /Zab From octopusfluff@REDACTED Wed Jul 6 12:00:58 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Wed, 6 Jul 2011 03:00:58 -0700 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 2:45 AM, Zabrane Mickael wrote: > > > Why measure this? Serving static content is one of the least > > interesting things I can think of > > to study. Any network worth the name won't bother to access the "root" > > server but will cache the content upstream. > > I disagree. That's not what the challenge is about. > Why don't just try it and avoid (useless) questions! These aren't useless questions. They're genuine concerns about the validity and applicability of your 'challenge'. As Joe notes, anything that doesn't have a short TTL can be presumed to be cached for at least some of the clients at least some of the time. In some cases, it could be all of the time for some of the clients; most significant ISPs make some attempt at this to limit outside bandwidth. Caching within the network is often cheaper, after all. For some time now, the useful places to be applying effort are non-trivial results, where the actual content being received is either in flux, or a direct result of the client's actions. You're wanting to measure something that's inaccurate, unnecessary, and uninformative. Could you give us a problem space defininition covering what exactly you're trying to achieve? From jesper.louis.andersen@REDACTED Wed Jul 6 12:43:19 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 6 Jul 2011 12:43:19 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: On Mon, Jul 4, 2011 at 17:11, Zabrane Mickael wrote: > The idea is ver simple: measure "how fast" you can serve a "100 bytes" file > (attached as "small.html") What are you trying to show with this benchmark? That is, what is the goal of it? Serving a static file is something nginx and a couple of Haskell HTTP servers are very good at (see the Warp-project). The main problem is that what you are trying to show might not be reflected very well from your test. > The rules are very simple : > 4. The HTTP client to use for this benchmark needs to be "Apache AB" > (http://httpd.apache.org/docs/2.0/programs/ab.html)?with: > $ ab -n 200000 -c 100 -t1 -k "http://127.0.0.1:8080/small.html" This violates Rule 1 in Mark Nottingham's http benchmark advice: http://www.mnot.net/blog/2011/05/18/http_benchmark_rules Quote: "The most common mistake I see people making is benchmarking a server on the same box where the load is generated. This doesn?t just put your results out a little bit, it makes them completely unreliable, because the load generator?s ?steal? of resources varies depending on how the server handles the load, which depends on resource availability." Is there any specific reason why you want to break that advice? Also, Steve Vinoski did a talk at the recent Erlang Factory on Yaws and the last part are benchmark and performance advice: http://www.erlang-factory.com/upload/presentations/409/vinoski-ef-london-2011.pdf I am with Joe here. Dynamically generated content is much more interesting for real problems. Static files can be outsourced to a CDN or Varnish easily. The reason you still serve them from the same webserver is convenience. -- J. From watson.timothy@REDACTED Wed Jul 6 12:58:45 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 6 Jul 2011 11:58:45 +0100 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: On 6 July 2011 01:52, Mike Oxford wrote: > erl running inside of a backgrounded emacs in the original directory > seems to have been the culprit. > Killed it off, nuked the directory, used the merged backup with the > rebuilt-nodes and things are good again. So that culprit was not rebar of anything to do with rebar? > > If two things are going to kill Erlang's uptake it's not going to be > the syntax, the performance, the functional-way-of-thinking... it'll > be the terrible error messages and the arcane build/packaging, both of > which make Erlang feel like a house of cards you can't even look at > wrong and, thus, give no confidence in actually running it in > production. ?(Not to mention just plain annoying devs to the point > they walk away from it, which is what kills more languages over time > than anything else.) So rebar is an effort to make a good build/packaging system for erlang. The problems you encountered don't seem to be directly related, as the aforementioned culprit was some other background process!? From kenneth.lundin@REDACTED Wed Jul 6 13:37:14 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 6 Jul 2011 13:37:14 +0200 Subject: [erlang-questions] function badarg In-Reply-To: References: <1176988592.58491309907479523.JavaMail.root@zimbra> Message-ID: In this case I think it is of most interest to trace on function calls. You will then be able to see the arguments that the loop function was called with just before the badarg crash and with that information you should be able to understand which function clause that was executing. In the next major release (R15B) planner for mid December there will be support for line numbers in crashes and it will be very easy to find exactly where you got the error. If you get a badarg it is most probably a call to a BIF that fails. /Kenneth Erlang/OTP, Ericsson On Wed, Jul 6, 2011 at 11:04 AM, Gleb Peregud wrote: > try using dbg module to set up tracing of received messages. This will > at least tell you which branch might have matched and executed. > > On Wed, Jul 6, 2011 at 01:11, Robert Virding > wrote: >> For the moment, no. It is said that in R15 errors will not just include in which function they occur but also on which line. You get a badarg error when you make a system call or call to OTP libraries and the arguments are wrong, wrong type or illegal value. >> >> Robert >> >> ----- "Alexander Kuleshov" wrote: >> >>> Hello, >>> >>> I have loop/2 function. This is main cycle of processing input data. >>> >>> Approximate form of this function: >>> >>> loop(Session, State) -> >>> {1} -> >>> ..... >>> loop(Session, State); >>> {2} -> >>> ..... >>> loop(Session, State); >>> {3} -> >>> ..... >>> loop(Session, State); >>> _ -> >>> loop(Session, State). >>> >>> In fact, this function big. Something about 1000 string of code. >>> >>> Sometimes this function crash with: Error in process <0.384.0> on >>> node >>> 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} >>> >>> Is there instruments or methods to know where it occurs and how it >>> can >>> catch this? >>> >>> Thank you. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zabrane3@REDACTED Wed Jul 6 13:42:19 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 6 Jul 2011 13:42:19 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: > What are you trying to show with this benchmark? That is, what is the > goal of it? At the end of the day, I'll try to compare Erlang Web servers with others competitors (not written in Erlang). > Serving a static file is something nginx and a couple of > Haskell HTTP servers are very good at (see the Warp-project). Please, just follow and stick to the rules. Nginx or Varnish (https://www.varnish-cache.org/) or any HTTP accelerator are out of this challenge scope. > This violates Rule 1 in Mark Nottingham's http benchmark advice: > http://www.mnot.net/blog/2011/05/18/http_benchmark_rules I'm aware of that rule (Steve Vinoski post: http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-benchmarking/). Thanks anyway. Why it's so hard to just play the game guys? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Wed Jul 6 13:50:13 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 06 Jul 2011 12:50:13 +0100 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: <4E144BF5.80000@llaisdy.com> On 06/07/2011 12:42, Zabrane Mickael wrote: >> What are you trying to show with this benchmark? That is, what is the >> goal of it? > > At the end of the day, I'll try to compare Erlang Web servers with others > competitors (not written in Erlang). > >> Serving a static file is something nginx and a couple of >> Haskell HTTP servers are very good at (see the Warp-project). > > Please, just follow and stick to the rules. > Nginx or Varnish (https://www.varnish-cache.org/) or any HTTP > accelerator are out of this challenge scope. Why is nginx outside of the challenge scope? If you want "to compare Erlang Web servers with other competitors (not written in Erlang)", surely nginx is one of the main competitors? Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From zabrane3@REDACTED Wed Jul 6 13:59:42 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 6 Jul 2011 13:59:42 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <4E144BF5.80000@llaisdy.com> References: <4E144BF5.80000@llaisdy.com> Message-ID: <48AC706D-3069-41E7-A4B5-3C787337246C@gmail.com> >> > Why is nginx outside of the challenge scope? Nginx or anything else are out of scope[1] at this time. This will not prevent you to write a simple cache in (pure) Erlang for the challenge. > If you want "to compare Erlang Web servers with other competitors (not written in Erlang)", surely nginx is one of the main competitors? Yes, but Nginx is written in C. [1] I'm planning to ask the authors of the other tools (like Cherokee, Apache2, Lighttpd + Nginx, Varnish, ...) and let them know about the challenge (with the same rules of course). -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Jul 6 14:08:16 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Jul 2011 14:08:16 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 1:42 PM, Zabrane Mickael wrote: > What are you trying to show with this benchmark? That is, what is the > goal of it? > > At the end of the day, I'll try to compare Erlang Web servers with others > competitors (not written in Erlang). > > Serving a static file is something nginx and a couple of > Haskell HTTP servers are very good at (see the Warp-project). > > Please, just follow and stick to the rules. > Nginx or Varnish (https://www.varnish-cache.org/)?or any HTTP accelerator > are out of this challenge scope. > > This violates Rule 1 in Mark Nottingham's http benchmark advice: > http://www.mnot.net/blog/2011/05/18/http_benchmark_rules > > I'm aware of that rule (Steve Vinoski > post:?http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-benchmarking/). > Thanks anyway. > Why it's so hard to just play the game guys? Because if you propose a new game then people will question and try to change the rules. That's how baseball got invented, and I know cricket is far better, but that is a bit off-topic here. In the particular case just about the last thing I want to know about is how fast I can serve static content - there is an entire industry devoted to exactly this question. Erlang was designed for building fault-tolerant systems, it was *not* designed for building fast systems. I would expect therefore that it would do badly at things that non-fault tolerant systems are good at (like serving static content). (( I'm sure we could build a fast content delivery network in Erlang - but this is not easy - the low hanging fruit was taken year ago - and it's got nothing to do with Erlang, its all about algorithms, caching, locality peering agreements etc.)) Like it or not, the results of such a measurement will be misinterpreted and abused. The fastest ever Erlang web server should be way slower than the fastest non-fault tolerant web server. A result of the form "the fastest Erlang web server is way slower than the fastest non fault-tolerant server" will be widely quoted and blogged, and the results will be taken out-of-context. For me I like to "make things work" before I "make them fast" - so the question of exactly what happens under stress, just before things fail is extremely interesting. I am very interested to know exactly how and when things fail when you stress them. Sending 100 bytes from a file seems to measure nothing interesting. Since it's the same file I expect it to be read *once* then cached in memory. It's so small that I expect no IP fragmentation at all. You won't really be measuring the performance of the web server, but rather the perfomance of a tiny sub-set of gen_tcp and file. If the authors of the various web serveres havn't made some catestripohuic mistales in their implamentaion I expect their code to be just a thin layer of the standard librarie rountines and I expect the results to be broadly similar. I always like to ask the question: What would you do with the results of such a measurement? Because unless you can give a good answer to this, there does not seem much point in doing the experiment. /joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From erlang@REDACTED Wed Jul 6 14:16:21 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Jul 2011 14:16:21 +0200 Subject: [erlang-questions] CALL FOR PAPERS Message-ID: There's a new ACM conference starting which is all about topics that quite a few of us find very interesting. This should be a good place to publish results on the use of Erlang for making distributed systems. I'm sure the conference would welcome a lot of "practical applications of ... " type papers. Here's your chance to tell the (academic) world about your wonderful Erlang applications Cheers /Joe Armstrong CALL FOR PAPERS AGERE! Programming Systems, Languages, and Applications based on Agents, Actors, and Decentralized Control http://agere2011.apice.unibo.it International Workshop to be held at SPLASH 2011, Portland, USA (http://splashcon.org/2011/) Sponsored by ACM SIGPLAN IMPORTANT DATES: Title and Abstract Submission: Friday, 5 August 2011 Paper Submission: Friday, 12 August 2011 Demo submission: Friday, 9 September 2011 Workshop: Monday, 24 October 2011 RESUME The fundamental turn of software into concurrency and distribution is not only a matter of performance, but also of design and abstraction, calling for programming paradigms that would allow more naturally than the current ones to think, design, develop, execute, debug and profile programs exhibiting different degrees of concurrency, reactiveness, autonomy, decentralization of control, distribution. This workshop aims at exploring programming approaches explicitly providing a level of abstraction that promotes a decentralized mindset in solving problems and programming systems. To this end, the abstractions of agents and actors (and systems of agents and actors) are taken as a natural reference: the objective of the workshop is then to foster the research in all aspects of agent-oriented programming and actor-oriented programming as evolution of mainstream paradigms (such as OOP), including the theory and the practice of design and programming, bringing together researchers working on the models, languages and technologies, and practitioners developing real-world systems and applications. Read more at: http://agere2011.apice.unibo.it "Start the workshop Now!" - join the group: https://groups.google.com/group/agere-at-splash to contribute to ongoing discussions about the AGERE! topics and workshop organization ORGANIZING COMMITTEE Gul Agha, University of Illinois at Urbana-Champaign, USA Rafael H. Bordini, Federal University of Rio Grande do Sul, Brazil Alessandro Ricci, University of Bologna, Italy PROGRAM COMMITTEE (in alphabetic order, to be completed) Gul Agha, University of Illinois at Urbana-Champaign, USA Joe Armstrong, Ericsson, Sweden Olivier Boissier, LSTI ENS Mines Saint-Etienne, France Rafael Bordini, Federal University of Rio Grande do Sul, Brazil Jean-Pierre Briot, LIP6, Paris 6, France Rem Collier, UCL, Dublin Mehdi Dastani, Utrecht University, The Netherlands Jurgen Dix, Technical University of Clausthal, Germany Koen Hindriks, Delft University of Technology, The Netherlands Jomi Hubner, Federal University of Santa Catarina, Brazil Joao Leite, New University of Lisbon, Portugal Jamali Nadeem, University of Saskatchewan, Saskatoon, Canada Ravi Pandya, Microsoft Jens Palsberg, UCLA, Los Angeles, USA Alessandro Ricci, University of Bologna, Italy Birna van Riemsdijk, Delft University of Technology, The Netherlands Giovanni Rimassa, Whitestein Technologies Amal El Fallah Seghrouchni, LIP6 - University Pierre and Marie Curie Munindar Singh, North Carolina State University, USA Akinori Yonezawa, University of Tokyo, Japan ... From ivan@REDACTED Wed Jul 6 14:18:41 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 06 Jul 2011 13:18:41 +0100 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <48AC706D-3069-41E7-A4B5-3C787337246C@gmail.com> References: <4E144BF5.80000@llaisdy.com> <48AC706D-3069-41E7-A4B5-3C787337246C@gmail.com> Message-ID: <4E1452A1.7000700@llaisdy.com> Most of the web server benchmarks I've come across on the web suffer from the same flaw: a partisan of server X tests a highly tuned and optimised install of their favourite server against default installs of competitor servers. These kinds of benchmarks are really not worth anything. I think it's a good idea to set a challenge, and invite server developers to show how their own project can meet the challenge. The first step is to devise a challenge, and this is not a trivial step. This step is actually listing the essential functional requirements of a web server. The ability to serve a static file promptly from disk is one requirement, but these days I shouldn't think it's the only or even the main one. I think it might be interesting to agree on a small set of requirements and then devise a challenge which could test those requirements. Best wishes Ivan On 06/07/2011 12:59, Zabrane Mickael wrote: >>> >> Why is nginx outside of the challenge scope? > > Nginx or anything else are out of scope[1] at this time. > This will not prevent you to write a simple cache in (pure) Erlang for > the challenge. > >> If you want "to compare Erlang Web servers with other competitors (not >> written in Erlang)", surely nginx is one of the main competitors? > > Yes, but Nginx is written in C. > > [1] I'm planning to ask the authors of the other tools (like Cherokee, > Apache2, Lighttpd + Nginx, Varnish, ...) > and let them know about the challenge (with the same rules of course). -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From erlang@REDACTED Wed Jul 6 14:40:19 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 6 Jul 2011 14:40:19 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <4E1452A1.7000700@llaisdy.com> References: <4E144BF5.80000@llaisdy.com> <48AC706D-3069-41E7-A4B5-3C787337246C@gmail.com> <4E1452A1.7000700@llaisdy.com> Message-ID: On Wed, Jul 6, 2011 at 2:18 PM, Ivan Uemlianin wrote: > Most of the web server benchmarks I've come across on the web suffer from > the same flaw: a partisan of server X tests a highly tuned and optimised > install of their favourite server against default installs of competitor > servers. ?These kinds of benchmarks are really not worth anything. > > I think it's a good idea to set a challenge, and invite server developers to > show how their own project can meet the challenge. > > The first step is to devise a challenge, and this is not a trivial step. > ?This step is actually listing the essential functional requirements of a > web server. ?The ability to serve a static file promptly from disk is one > requirement, but these days I shouldn't think it's the only or even the main > one. I know that sounds fine, but it's the non-functional requirements that always seem to make or break a project. How good is the documentation? How mature is the product? How easy is it to maintain? Can I understand how the thing works? Is it covered by the blue-river-snowy public license? ... The answers to these questions are just not quantifiable. Not being covered by the blue-river-snowy license and having no documentation would be a very good reason not to use a web server even if it could serve pages faster than a shark can bite off your leg. /Joe > > I think it might be interesting to agree on a small set of requirements and > then devise a challenge which could test those requirements. > > Best wishes > > Ivan > > > On 06/07/2011 12:59, Zabrane Mickael wrote: >>>> >>> Why is nginx outside of the challenge scope? >> >> Nginx or anything else are out of scope[1] at this time. >> This will not prevent you to write a simple cache in (pure) Erlang for >> the challenge. >> >>> If you want "to compare Erlang Web servers with other competitors (not >>> written in Erlang)", surely nginx is one of the main competitors? >> >> Yes, but Nginx is written in C. >> >> [1] I'm planning to ask the authors of the other tools (like Cherokee, >> Apache2, Lighttpd + Nginx, Varnish, ...) >> and let them know about the challenge (with the same rules of course). > > > -- > ============================================================ > Ivan A. Uemlianin > Speech Technology Research and Development > > ? ? ? ? ? ? ? ? ? ?ivan@REDACTED > ? ? ? ? ? ? ? ? ? ? www.llaisdy.com > ? ? ? ? ? ? ? ? ? ? ? ? llaisdy.wordpress.com > ? ? ? ? ? ? ? ? ? ? www.linkedin.com/in/ivanuemlianin > > ? ?"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > ? ? ? ? ? ? ? ? ? ? (Schiller, Beethoven) > ============================================================ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dave@REDACTED Wed Jul 6 14:45:35 2011 From: dave@REDACTED (David Goehrig) Date: Wed, 6 Jul 2011 08:45:35 -0400 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: <4AE4A15E-BE09-41D9-AECA-701076D44DEC@nexttolast.com> On Jul 6, 2011, at 5:45 AM, Zabrane Mickael wrote: > > I disagree. That's not what the challenge is about. > Why don't just try it and avoid (useless) questions! Because your "benchmark" is meaningless without proper controls and variables. This supposed challenge characterizes nothing, because you've not defined any meaningful constraints on the system. A meaningful benchmark measures changes in response to one or more environmental pressures. For example, you can measure OS process starvation's effect on tcp backlog across each server with memory, io, CPU load, and payload all remaining constant. Similarly you can look at performance with respect to slow connections, high packet loss, CPU starvation, memory starvation, file descriptor starvation, and io blocking. And just about any other variable you can control for. Throughput will vary as a function of all these factors and will demonstrate a lot of important issues for each system design. So to answer your question, the best reason not to try the challenge is because it is a terrible waste of time that could be better spent properly characterizing a production system. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jul 6 15:46:21 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 6 Jul 2011 15:46:21 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 13:42, Zabrane Mickael wrote: > What are you trying to show with this benchmark? That is, what is the > goal of it? > > At the end of the day, I'll try to compare Erlang Web servers with others > competitors (not written in Erlang). That was not the answer I had in mind when phrasing the question. I'll try to be more precise: What are you going to measure? If you are going to count requests, then the problem is that a 100 bytes transfer is way less than the size of the average HTTP header. In other words, you will be measuring who cheats and does not provide full headers as per the HTTP spec. If you are going to measure throughput, on a single machine, then 'ab' and whatever web server you have will fight for the resources and thus show nothing at all. What I am after is why you think your benchmark will give you any indication of the goal you had in mind. What is performance to you? I am interested in your answer because it is important to me if I want to use the results for anything. It may be flawed or not, that is not the concern for me. I am interested in the reason for why you laid out the rules of the game as your did. I disagree that you can see web servers as competitors to each other. There is always a greater purpose behind writing a web server. From the point where it is a personal exercise (can I do it?), to providing a web server in your favorite language for ease of integration, to writing a server because you want to earn money. It is not as much a competition as it is a convenience of use. Or said otherwise, many of the competitors are not even aware they are competing :) > I'm aware of that rule (Steve Vinoski > post:?http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-benchmarking/). > Thanks anyway. What are you going to do about it? Those who will have multiple CPUs in their machine and pins 'ab' to one and gives Erlang free reign over the others will definitely be at an advantage here. In fact, those who disable SMP from the Erlang runtime might also be faster. If you ran many tests and then used a bootstrapping technique, then you might be able to tell if the localhost test generator tool interferes too much with the results for instance. That is one way to circumvent that problem. > Why it's so hard to just play the game guys? We are playing the game on another level. Our game is "The scientific method and experimental testing". To be precise, I think your test has way to many confounding factors to underpin your hypothesis that your experiment can be used to show anything about web servers in comparison to each other. When you design a simple experiment, it is mandatory in our game to point out exactly what the confounding factors are and what trouble there will be when drawing conclusions from the test itself. I have pointed out some factors and Joe has pointed out some different factors. That is all. -- J. From cmcknight@REDACTED Wed Jul 6 17:30:18 2011 From: cmcknight@REDACTED (cmcknight@REDACTED) Date: Wed, 6 Jul 2011 08:30:18 -0700 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: Message-ID: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> At the risk of being offensive, your challenge seems to have no technical merit and telling people to "just stick to the rules" will likely garner you no participants. Sever people have asked why you want to measure this sort of system where, as Joe points out, the size of data ensures it will be cached in a real system. There are a number of other interesting problems where Erlang is an ideal answer because of its fault-tolerant nature and relative ease in building / managing distributed systems. Speed, as Joe noted, is not necessarily Erlang's strongest point. It's "fast enough", and for problem that lend themselves to parallelization, is interesting. If your point is to try to get people to go to a lot of effort to create a meaningless benchmark, then I wish you luck but don't be surprised at the questions, pushback or general lack of participation, especially with the tone you are using in your messages. Sent from my iPhone 3GS On Jul 6, 2011, at 3:00 AM, erlang-questions-request@REDACTED wrote: > Send erlang-questions mailing list submissions to > erlang-questions@REDACTED > > To subscribe or unsubscribe via the World Wide Web, visit > http://erlang.org/mailman/listinfo/erlang-questions > or, via email, send a message with subject or body 'help' to > erlang-questions-request@REDACTED > > You can reach the person managing the list at > erlang-questions-owner@REDACTED > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of erlang-questions digest..." > > > Today's Topics: > > 1. Re: Erlang Web Servers challenge (Zabrane Mickael) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 6 Jul 2011 11:45:50 +0200 > From: Zabrane Mickael > To: Joe Armstrong > Cc: Erlang-Questions Questions > Subject: Re: [erlang-questions] Erlang Web Servers challenge > Message-ID: > Content-Type: text/plain; charset=us-ascii > >> Why measure this? Serving static content is one of the least >> interesting things I can think of >> to study. Any network worth the name won't bother to access the "root" >> server but will cache the content upstream. > > I disagree. That's not what the challenge is about. > Why don't just try it and avoid (useless) questions! > >> How fast you can serve up dynamic content is far more interesting, >> since it can't be cached. > > This will be part of my second challenge of course. > For now, let's stick with the first challenge rules please. > > Regards > /Zab > > > ------------------------------ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > End of erlang-questions Digest, Vol 16, Issue 13 > ************************************************ From dmercer@REDACTED Wed Jul 6 18:08:12 2011 From: dmercer@REDACTED (David Mercer) Date: Wed, 6 Jul 2011 11:08:12 -0500 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: <00f001cc3bf6$e94f3e60$bbedbb20$@com> On Wednesday, July 06, 2011, Joe Armstrong wrote: > Erlang was designed for building fault-tolerant systems, it was *not* > designed > for building fast systems +1. Salient point worth repeating. This often gets forgotten. > The fastest ever Erlang web server should be way slower than the > fastest non-fault > tolerant web server. Cheers, DBM From moxford@REDACTED Wed Jul 6 20:01:22 2011 From: moxford@REDACTED (Mike Oxford) Date: Wed, 6 Jul 2011 11:01:22 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: On Wed, Jul 6, 2011 at 3:58 AM, Tim Watson wrote: > So that culprit was not rebar of anything to do with rebar? --snip > So rebar is an effort to make a good build/packaging system for > erlang. The problems you encountered don't seem to be directly > related, as the aforementioned culprit was some other background > process!? > The issue was -caused- by the rebar generation pass; getting back -out- was hindered by the backgrounded process. As it stands, something is borked because that directory is now "corrupted" somehow. I nuke the directory and do a full rebuild and the .ez file does not have the .beam files...only the .app file. I can now compile and run perfectly in any other directory....just not that one. And I've rebooted and nuked the filesystem tree. It's very strange. It's like something is pattern-matching on the directory structure/name and causing problems. FWIW, I really ke erlang's syntax ... and my comments above were directed more towards build/release than erlang proper. Rebar does an admirable job of pulling things together but even though Erlang's been around a while it's not had the level of tools-maturity that other platforms have had. (Clarification there: They're mature if you're in-the-know and have learned the arcane but they're not terribly user-friendly which is a major obstacle to widespread acceptance.) Perhaps, as mentioned above, the correct course for me is to use make for more than just front-ending rebar. However, while fluent in make, that requires a deeper dive into the build/release process which then pushes back on project release timelines. :) -mox From jwatte@REDACTED Wed Jul 6 21:32:41 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 6 Jul 2011 12:32:41 -0700 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: Actually, test driven development methodology has a very clear separation between these: - acceptance tests / interface tests use only the public, exported interface of a component/system/unit, and tests it in a "clean room" environment. - unit tests grope around the internals of your unit in a way that nobody on the outside can. It's used to verify implementation details of the unit that may be important to the implementation, but not to anyone on the outside. There is a little bit of religion in this -- some people tend to feel very strongly that only acceptance tests matter. Personally, I find unit tests to be useful in many cases, especially when growing an implementation using testing in the first place. For example: You might want to test a gen_server. This gen_server uses a state record that is internal to the module. A unit test, inside the module, can set up a state record, and then call handle_call() directly, making sure that particular implementation details do what they're supposed to. Meanwhile, nobody outside the module should ever call that function (except for the gen_server itself, of course), and any test on the outside should only use the public interface functions. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 5, 2011 at 10:56 PM, Gianfranco Alongi < gianfranco.alongi@REDACTED> wrote: > I disagree. > Putting your tests in another module will actually force you to think more > of your design, making the application/lib more testable without being > necessarily more "open". Another huge win is that you will have a cleaner > implementation module. And you will be able to easily mix and match > revisions of code and tests. /G > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 6 21:38:50 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 6 Jul 2011 12:38:50 -0700 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> Message-ID: Another ambiguous part of the problem as stated: The way the question is worded, I could write a TCP server that accepts a connection, then spits out a static blob of text consisting of static HTTP headers and the contents of the file -- I don't even need to parse the request, because this file is the only thing that the server will serve. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jul 6, 2011 at 8:30 AM, cmcknight@REDACTED wrote: > At the risk of being offensive, your challenge seems to have no technical > merit and telling people to "just stick to the rules" will likely garner you > no participants. Sever people have asked why you want to measure this sort > of system where, as Joe points out, the size of data ensures it will be > cached in a real system. There are a number of other interesting problems > where Erlang is an ideal answer because of its fault-tolerant nature and > relative ease in building / managing distributed systems. Speed, as Joe > noted, is not necessarily Erlang's strongest point. It's "fast enough", and > for problem that lend themselves to parallelization, is interesting. If your > point is to try to get people to go to a lot of effort to create a > meaningless benchmark, then I wish you luck but don't be surprised at the > questions, pushback or general lack of participation, especially with the > tone you are using in your messages. > > > > > > Sent from my iPhone 3GS > > On Jul 6, 2011, at 3:00 AM, erlang-questions-request@REDACTED wrote: > > > Send erlang-questions mailing list submissions to > > erlang-questions@REDACTED > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://erlang.org/mailman/listinfo/erlang-questions > > or, via email, send a message with subject or body 'help' to > > erlang-questions-request@REDACTED > > > > You can reach the person managing the list at > > erlang-questions-owner@REDACTED > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of erlang-questions digest..." > > > > > > Today's Topics: > > > > 1. Re: Erlang Web Servers challenge (Zabrane Mickael) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Wed, 6 Jul 2011 11:45:50 +0200 > > From: Zabrane Mickael > > To: Joe Armstrong > > Cc: Erlang-Questions Questions > > Subject: Re: [erlang-questions] Erlang Web Servers challenge > > Message-ID: > > Content-Type: text/plain; charset=us-ascii > > > >> Why measure this? Serving static content is one of the least > >> interesting things I can think of > >> to study. Any network worth the name won't bother to access the "root" > >> server but will cache the content upstream. > > > > I disagree. That's not what the challenge is about. > > Why don't just try it and avoid (useless) questions! > > > >> How fast you can serve up dynamic content is far more interesting, > >> since it can't be cached. > > > > This will be part of my second challenge of course. > > For now, let's stick with the first challenge rules please. > > > > Regards > > /Zab > > > > > > ------------------------------ > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > End of erlang-questions Digest, Vol 16, Issue 13 > > ************************************************ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Wed Jul 6 21:47:31 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 6 Jul 2011 21:47:31 +0200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> Message-ID: Le 6 juil. 2011 ? 21:38, Jon Watte a ?crit : > Another ambiguous part of the problem as stated: > The way the question is worded, I could write a TCP server that accepts a connection, then spits out a static blob of text consisting of static HTTP headers and the contents of the file -- I don't even need to parse the request, because this file is the only thing that the server will serve. Funny analysis, but that's a toy web server. In the challenge list, I asked for a real web server instead (I've cited Misultin, Yaws ...). From gianfranco.alongi@REDACTED Wed Jul 6 21:52:49 2011 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Wed, 6 Jul 2011 20:52:49 +0100 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: I believe the discussion started on where the tests should go? Anyway, test driven development is about driving the development of your code based on the tests. In order to get safe-holds and be able to stop at any time, knowing exactly the code base is able/not able to do. As Jon says, this is a topic in which some people tend to get zealous. I for one do not care whether your tests are on one part of the scale or the other. Really. If you create a battery of tests, which for all intents and purposes test and prove that the functional behaviour is sound, and you are happy with that. Great! Then it's going to be (hopefully) easy for you to change implementation details without carrying too much overhead. Down side - this will tend to give you longer leap-times when moving between Red (tests-failing) and Green (tests-working) states. Focusing heavily on the Unit testing perspective, you will have shorter cycles between Red and Green, a perceived higher commit-velocity and most robustness, while also carrying more overhead if you start mutating the implementation details quickly. Cheers /G On Wed, Jul 6, 2011 at 8:32 PM, Jon Watte wrote: > Actually, test driven development methodology has a very clear separation > between these: > - acceptance tests / interface tests use only the public, exported > interface of a component/system/unit, and tests it in a "clean room" > environment. > - unit tests grope around the internals of your unit in a way that nobody > on the outside can. It's used to verify implementation details of the unit > that may be important to the implementation, but not to anyone on the > outside. > > There is a little bit of religion in this -- some people tend to feel very > strongly that only acceptance tests matter. Personally, I find unit tests to > be useful in many cases, especially when growing an implementation using > testing in the first place. > > For example: You might want to test a gen_server. This gen_server uses a > state record that is internal to the module. A unit test, inside the module, > can set up a state record, and then call handle_call() directly, making sure > that particular implementation details do what they're supposed to. > Meanwhile, nobody outside the module should ever call that function (except > for the gen_server itself, of course), and any test on the outside should > only use the public interface functions. > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Tue, Jul 5, 2011 at 10:56 PM, Gianfranco Alongi < > gianfranco.alongi@REDACTED> wrote: > >> I disagree. >> Putting your tests in another module will actually force you to think more >> of your design, making the application/lib more testable without being >> necessarily more "open". Another huge win is that you will have a cleaner >> implementation module. And you will be able to easily mix and match >> revisions of code and tests. /G >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Wed Jul 6 21:55:39 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 6 Jul 2011 15:55:39 -0400 Subject: [erlang-questions] Unit testing code - whence? In-Reply-To: References: Message-ID: I think it's important to first define the complexity of a module. You have some very simple modules (or rather, simple in principle) with a functional interface. These can often be unit tested from the outside only, without bothering too much with the rest. In the case of these 'simple' modules, one of the problems with the tests that look at the innards of the module is that they can quickly become a maintenance overhead, and are also frequently useless in the face of refactoring -- you need to drop these tests, write new ones, etc. I find that I tend to rely on the interface I've established for all of these, instead. TDD done starting from the outside also has the advantage of having you thinking as the user of the code rather than the writer of it; a rather important point in design, I'd say. In the case of more complex modules, like gen_servers, which actually hide state and give you a destructive interface, I first tended to test things from the inside, but quickly decided that it was much simpler for me to test things from the outside, while cheating and using 'sys:get_status/1' to get a peek at the inside state of the server. That way, the whole thing remains more authentic to its runtime environment while still allowing you to do white box testing. It's also much simpler when you get side-effecting functions as part of the code itself (trying to send exit signals, links, gen_server:reply, etc.) In the case of even more complex processes, like FSMs with complex states where you don't want to repeat sequences of events leading to a given state, you can use gen_fsm:enter_loop/4 (http://www.erlang.org/documentation/doc-5.7.1/lib/stdlib-1.16.1/doc/html/gen_fsm.html#enter_loop-4) with a known state to observe the transitions and replies you need, although I've rarely done that myself, given how dirty it feels. I'll rather build the whole sequence in a setup function as it will also act as documentation for those who want to use that. Most of the time, it doesn't exactly matter to me where the limit is between unit and integration and system and etc. I usually just want to make sure things are tested right, and in the way that will bring me the least maintenance overhead possible when re-working code in a few months/years. Your mileage may vary depending on what kind of projects or within what kind of team you're working, though. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-06, at 15:32 PM, Jon Watte wrote: > Actually, test driven development methodology has a very clear separation between these: > - acceptance tests / interface tests use only the public, exported interface of a component/system/unit, and tests it in a "clean room" environment. > - unit tests grope around the internals of your unit in a way that nobody on the outside can. It's used to verify implementation details of the unit that may be important to the implementation, but not to anyone on the outside. > > There is a little bit of religion in this -- some people tend to feel very strongly that only acceptance tests matter. Personally, I find unit tests to be useful in many cases, especially when growing an implementation using testing in the first place. > > For example: You might want to test a gen_server. This gen_server uses a state record that is internal to the module. A unit test, inside the module, can set up a state record, and then call handle_call() directly, making sure that particular implementation details do what they're supposed to. Meanwhile, nobody outside the module should ever call that function (except for the gen_server itself, of course), and any test on the outside should only use the public interface functions. > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > On Tue, Jul 5, 2011 at 10:56 PM, Gianfranco Alongi wrote: > I disagree. > Putting your tests in another module will actually force you to think more of your design, making the application/lib more testable without being necessarily more "open". Another huge win is that you will have a cleaner implementation module. And you will be able to easily mix and match revisions of code and tests. /G > > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 6 22:05:22 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 6 Jul 2011 13:05:22 -0700 Subject: [erlang-questions] Mysterious boot failure Message-ID: We're running a cluster of 11 Erlang nodes, running on R13B3 using Ubuntu 10.04 LTS 64-bit. The application is installed in a subdirectory of the home directory of a particular user. We start the application using nohup, and with stdio/stderr re-direct to a log file. We're seeing a mysterious crash in some cases (see below) If we're logged in as the appropriate user, everything starts fine. If we're logged in as root, and do "sudo -u username " then beam.smp crashes with "init terminating in do_boot". Why is this? I can't figure out why it's crashing like this. {no error logger present) error: "Error in process <0.2.0> with exit value: {badarg,[{erl_prim_loader,check_file_result,3},{erl_prim_loader,check_file_result,3},{init,get_boot,1},{init,get_boot,2},{init,do_boot,3}]}\n" init terminating in do_boot () Googling on this error shows only two other web resources; one an IRC chat with basho, not getting an answer, and one a message on this mailing list, also not getting an answer. Reading the code, it seems as if the only way this will happen is if there is an error, but the "Func" argument or "Reason" argument to check_file_result is somehow not an atom. I don't, however, see how that could be happening. The other question is also why the boot script would not be visible/available at that point. The particular user has no .bashrc or other such init script. The actual start script (with secrets sanitized) is: nohup erl +K true -noshell \ -env ERL_MAX_PORTS 200500 \ +W w +P 1001001 \ -boot start_sasl \ -cluster leader@REDACTED \ -kernel inet_dist_listen_min 50000 inet_dist_listen_max 50009 \ -name "$node@$hostname" -setcookie "some secret" \ -callout svc_url '"http://service/%%.php"' \ -s launcher -extra $role \ "$logdir/$node-logfile.txt" 2>&1 & Here's the crash-dump analysis (not very useful): Slogan init terminating in do_boot () Node name 'mqnode@REDACTED' Crashdump created on Wed Jul 6 10:23:58 2011 System version Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:true] Compiled Fri Apr 9 12:29:55 2010 Memory allocated 141146456 bytes Atoms 5690 Processes 35 ETS tables 17 Funs 357 -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Wed Jul 6 22:29:31 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 6 Jul 2011 17:29:31 -0300 Subject: [erlang-questions] Mysterious boot failure In-Reply-To: References: Message-ID: I don't know if this is the cause, but if you're setting specific values for the system limits in /etc/security/limits.conf for your user, make sure that they're being set by PAM when executing via sudo. You can check this in /etc/pam.d/sudo. You should look for a line like the following one and make sure that it's not commented out: session required pam_limits.so Also, I'd also check that you're starting the Erlang VM in the correct directory (i.e. one where your user has permissions to read). Juanjo On Wed, Jul 6, 2011 at 5:05 PM, Jon Watte wrote: > We're running a cluster of 11 Erlang nodes, running on R13B3 using Ubuntu > 10.04 LTS 64-bit. > The application is installed in a subdirectory of the home directory of a > particular user. We start the application using nohup, and with stdio/stderr > re-direct to a log file. > We're seeing a mysterious crash in some cases (see below) > > If we're logged in as the appropriate user, everything starts fine. > If we're logged in as root, and do "sudo -u username " then > beam.smp crashes with "init terminating in do_boot". > Why is this? I can't figure out why it's crashing like this. > > {no error logger present) error: "Error in process <0.2.0> with exit value: > {badarg,[{erl_prim_loader,check_file_result,3},{erl_prim_loader,check_file_result,3},{init,get_boot,1},{init,get_boot,2},{init,do_boot,3}]}\n" > init terminating in do_boot () > > > Googling on this error shows only two other web resources; one an IRC chat > with basho, not getting an answer, and one a message on this mailing list, > also not getting an answer. > Reading the code, it seems as if the only way this will happen is if there > is an error, but the "Func" argument or "Reason" argument to > check_file_result is somehow not an atom. I don't, however, see how that > could be happening. > The other question is also why the boot script would not be > visible/available at that point. The particular user has no .bashrc or other > such init script. > > > The actual start script (with secrets sanitized) is: > > nohup erl +K true -noshell \ > -env ERL_MAX_PORTS 200500 \ > +W w +P 1001001 \ > -boot start_sasl \ > -cluster leader@REDACTED \ > -kernel inet_dist_listen_min 50000 inet_dist_listen_max 50009 \ > -name "$node@$hostname" -setcookie "some secret" \ > -callout svc_url '"http://service/%%.php"' \ > -s launcher -extra $role \ > "$logdir/$node-logfile.txt" 2>&1 & > > > > Here's the crash-dump analysis (not very useful): > > Slogan init terminating in do_boot () > Node name 'mqnode@REDACTED' > Crashdump created on Wed Jul 6 10:23:58 2011 > System version Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] > [rq:8] [async-threads:0] [hipe] [kernel-poll:true] > Compiled Fri Apr 9 12:29:55 2010 > Memory allocated 141146456 bytes > Atoms 5690 > Processes 35 > ETS tables 17 > Funs 357 > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Wed Jul 6 22:55:13 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Wed, 6 Jul 2011 13:55:13 -0700 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> Message-ID: On Wed, Jul 6, 2011 at 12:47 PM, Zabrane Mickael wrote: > > Le 6 juil. 2011 ? 21:38, Jon Watte a ?crit : > >> Another ambiguous part of the problem as stated: >> The way the question is worded, I could write a TCP server that accepts a connection, then spits out a static blob of text consisting of static HTTP headers and the contents of the file -- I don't even need to parse the request, because this file is the only thing that the server will serve. > > Funny analysis, but that's a toy web server. In the challenge list, I asked for a real web server instead (I've cited Misultin, Yaws ...). A toy web server for a toy benchmark. Seems like a perfect match. Again: You need to define your objectives here, or we're not going to get anywhere. If your objective is simply "I want to test webservers according to arbitrary criteria that are not up for discussion", then I think we're done here. From watson.timothy@REDACTED Wed Jul 6 23:51:50 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 6 Jul 2011 22:51:50 +0100 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: > The issue was -caused- by the rebar generation pass; getting back > -out- was hindered by the backgrounded process. > Ah ok, I didn't realise that at first. > As it stands, something is borked because that directory is now > "corrupted" somehow. ?I nuke the directory and do a full rebuild and > the .ez file does not have the .beam files...only the .app file. > I can now compile and run perfectly in any other directory....just not > that one. ?And I've rebooted and nuked the filesystem tree. > > It's very strange. ?It's like something is pattern-matching on the > directory structure/name and causing problems. > Does seem very weird. Have you submitted this to the rebar mailing list for analysis? I haven't seen any posts. Also submitting a github issue with the output from `rebar generate -v` would be useful if that's a possibility. > FWIW, I really ke erlang's syntax ... and my comments above were > directed more towards build/release than erlang proper. ?Rebar does an > admirable job of pulling things together but even though Erlang's been > around a while it's not had the level of tools-maturity that other > platforms have had. ?(Clarification there: They're mature if you're > in-the-know and have learned the arcane but they're not terribly > user-friendly which is a major obstacle to widespread acceptance.) Yes I agree that the tools support is a little lacking. Thankfully Erlang is simple enough that this isn't often a problem, but when you do run into issues with the tools it can be a real pain to isolate and fix them. > > Perhaps, as mentioned above, the correct course for me is to use make > for more than just front-ending rebar. ?However, while fluent in make, > that requires a deeper dive into the build/release process which then > pushes back on project release timelines. ?:) > Yeah indeed - make is good and all but few people would dream of building a java project without ant or maven (or even gradle and what have you) so having a tool that "does the right thing" most of the time and can be extended when required is a very good thing IMO. Even C/C++ projects can benefit from more user friendly tools these days. I seriously think engaging with the rebar mailing list and talking to the developers would be worthwhile - they're very helpful and there's a lot of knowledge and experience on the mailing list especially around the release generation and app-upgrade features. From g@REDACTED Thu Jul 7 00:35:21 2011 From: g@REDACTED (Garrett Smith) Date: Wed, 6 Jul 2011 16:35:21 -0600 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <00f001cc3bf6$e94f3e60$bbedbb20$@com> References: <00f001cc3bf6$e94f3e60$bbedbb20$@com> Message-ID: Interesting thread. Reminds me of a conversation I overhead, in different time and place: http://www.youtube.com/watch?v=xEJ1n13soWU Very slightly NSFW language. Garrett P.S. I hope this isn't viewed as piling on. I just couldn't resist. This is absolutely not personally directed. It's just a little fun poking at a certain ideology that we've all been guilty of adopting :) On Wed, Jul 6, 2011 at 10:08 AM, David Mercer wrote: > On Wednesday, July 06, 2011, Joe Armstrong wrote: > >> Erlang was designed for building fault-tolerant systems, it was *not* >> designed >> for building fast systems > > +1. ?Salient point worth repeating. ?This often gets forgotten. > >> The fastest ever Erlang web server should be way slower than the >> fastest non-fault >> tolerant web server. > > Cheers, > > DBM > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From peralta.alejandro@REDACTED Thu Jul 7 04:37:42 2011 From: peralta.alejandro@REDACTED (Ale) Date: Wed, 6 Jul 2011 23:37:42 -0300 Subject: [erlang-questions] Asynchronous I/O - Getting documentation. Message-ID: Hello All, I'm looking for documentation to expand my knowledge of the implementation of how asynchronous I/O works; from wikipedia. """This approach is also used in the Erlang programming language runtime system. The Erlang virtual machine uses asynchronous IO using a small pool of only a few threads or sometimes just one process, to handle IO from up to millions of Erlang processes. IO handling in each process is written mostly using blocking synchronous I/O. This way high performance of asynchronous I/O is merged with simplicity of normal IO. Many IO problems in Erlang are mapped to message passing, which can be easily processed using built-in selective receive.""" I'm interested in learning how the scheduler works, and how the process which handle IO work. Thanks, -- Ale. From steven.charles.davis@REDACTED Thu Jul 7 05:02:10 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 6 Jul 2011 20:02:10 -0700 (PDT) Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: Message-ID: <0af17901-69f6-4f3e-99ae-fa40477fc988@z12g2000yqj.googlegroups.com> I'm thinking that the real comparison that everyone would like to see (obviously) would be to build a full web application infrastructure using various current technologies and with all the regression bases covered and major stress tests.... for an app like facebook. Of course, you'd need to use the best expertise from each technology domain to make sure the comparison was even slightly "fair". And of course, this would require such a monumental investment it could never be done (what's more most all platforms never manage to test themselves until put into the wild). So we are left making value judgements over and over again. So I'm thinking: What exactly is the value of this test beyond serving favicons (i'd choose nginx for that probably)? My negative 2c. :( /s On Jul 4, 10:11?am, Zabrane Mickael wrote: > Hi guys, > > I'd like your help for this small Web servers challenge. > > The idea is ver simple: measure "how fast" you can serve a "100 bytes" file (attached as "small.html") > in "less than 01 second" by using one of the following web servers: > > - Yaws (http://yaws.hyber.org/download/yaws-1.90.tar.gz) > - Mochiweb (https://github.com/mochi/mochiweb) > - Misultin (https://github.com/ostinelli/misultin) > - Cowboy (https://github.com/extend/cowboy) > - Bare metal web server using "gen_tcp" > > The rules are very simple : > > 1. The first time you served the file, you MUST read it from disk. > All subsequent requests to that file can use any caching mechanism! > > 2. Any (hidden) trick is allowed (eg. prim_inet, SMP enabled ...) > > 3. Erlang versions R14B to R14B03 > > 4. The HTTP client to use for this benchmark needs to be "Apache AB" (http://httpd.apache.org/docs/2.0/programs/ab.html) with: > $ ab -n 200000 -c 100 -t1 -k "http://127.0.0.1:8080/small.html" > > I hope that the authors if these web servers will also contribute to this bench as they > know their software better than anybody else. > > Challenge's open ;-) > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Thu Jul 7 06:31:08 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 7 Jul 2011 06:31:08 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: <00f001cc3bf6$e94f3e60$bbedbb20$@com> Message-ID: <255278E5-BF8E-46DA-A9EF-C77F195A769D@gmail.com> Le 7 juil. 2011 ? 00:35, Garrett Smith a ?crit : > Interesting thread. Reminds me of a conversation I overhead, in > different time and place: > > http://www.youtube.com/watch?v=xEJ1n13soWU ;-) From erlang@REDACTED Thu Jul 7 08:34:33 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Jul 2011 08:34:33 +0200 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: <00f001cc3bf6$e94f3e60$bbedbb20$@com> Message-ID: On Thu, Jul 7, 2011 at 12:35 AM, Garrett Smith wrote: > Interesting thread. Reminds me of a conversation I overhead, in > different time and place: > > http://www.youtube.com/watch?v=xEJ1n13soWU > > Very slightly NSFW language. Wonderfull - any ideas how this movie was made? /Joe > > Garrett > > P.S. I hope this isn't viewed as piling on. I just couldn't resist. > This is absolutely not personally directed. It's just a little fun > poking at a certain ideology that we've all been guilty of adopting :) > > On Wed, Jul 6, 2011 at 10:08 AM, David Mercer wrote: >> On Wednesday, July 06, 2011, Joe Armstrong wrote: >> >>> Erlang was designed for building fault-tolerant systems, it was *not* >>> designed >>> for building fast systems >> >> +1. ?Salient point worth repeating. ?This often gets forgotten. >> >>> The fastest ever Erlang web server should be way slower than the >>> fastest non-fault >>> tolerant web server. >> >> Cheers, >> >> DBM >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From ivan@REDACTED Thu Jul 7 09:53:57 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 7 Jul 2011 08:53:57 +0100 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: References: <00f001cc3bf6$e94f3e60$bbedbb20$@com> Message-ID: <86989A08-AEB3-44F7-89EB-EE0C78F9AF87@llaisdy.com> +2! And the synthetic voices aren't bad either. xtranormal looks pretty cool. Ivan Sent from iPhone. On 7 Jul 2011, at 07:34, Joe Armstrong wrote: > On Thu, Jul 7, 2011 at 12:35 AM, Garrett Smith wrote: >> Interesting thread. Reminds me of a conversation I overhead, in >> different time and place: >> >> http://www.youtube.com/watch?v=xEJ1n13soWU >> >> Very slightly NSFW language. > > Wonderfull - any ideas how this movie was made? > > /Joe From ola.a.andersson@REDACTED Thu Jul 7 10:16:14 2011 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Thu, 7 Jul 2011 10:16:14 +0200 Subject: [erlang-questions] function badarg In-Reply-To: References: <1176988592.58491309907479523.JavaMail.root@zimbra> Message-ID: Great news with the line number in the crash. That will reduce my user support activities a lot. Thanx. ;-) Regarding the original question I think that you should really consider some refactoring if you have a function with 1000 loc. I remember an old rule of thumb saying that a MODULE should not contain more than about 400 loc. Personally I seldom use more than 10 lines in any function. That more or less eliminates any badmatches and it makes it that much easier to debug. /OLA. -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Kenneth Lundin Sent: den 6 juli 2011 13:37 To: Gleb Peregud Cc: erlang-questions Subject: Re: [erlang-questions] function badarg In this case I think it is of most interest to trace on function calls. You will then be able to see the arguments that the loop function was called with just before the badarg crash and with that information you should be able to understand which function clause that was executing. In the next major release (R15B) planner for mid December there will be support for line numbers in crashes and it will be very easy to find exactly where you got the error. If you get a badarg it is most probably a call to a BIF that fails. /Kenneth Erlang/OTP, Ericsson On Wed, Jul 6, 2011 at 11:04 AM, Gleb Peregud wrote: > try using dbg module to set up tracing of received messages. This will > at least tell you which branch might have matched and executed. > > On Wed, Jul 6, 2011 at 01:11, Robert Virding > wrote: >> For the moment, no. It is said that in R15 errors will not just include in which function they occur but also on which line. You get a badarg error when you make a system call or call to OTP libraries and the arguments are wrong, wrong type or illegal value. >> >> Robert >> >> ----- "Alexander Kuleshov" wrote: >> >>> Hello, >>> >>> I have loop/2 function. This is main cycle of processing input data. >>> >>> Approximate form of this function: >>> >>> loop(Session, State) -> >>> {1} -> >>> ..... >>> loop(Session, State); >>> {2} -> >>> ..... >>> loop(Session, State); >>> {3} -> >>> ..... >>> loop(Session, State); >>> _ -> >>> loop(Session, State). >>> >>> In fact, this function big. Something about 1000 string of code. >>> >>> Sometimes this function crash with: Error in process <0.384.0> on >>> node 'test@REDACTED' with exit value: {badarg,[{test,loop,2}]} >>> >>> Is there instruments or methods to know where it occurs and how it >>> can catch this? >>> >>> Thank you. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From jodie.burch@REDACTED Thu Jul 7 12:00:46 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Thu, 07 Jul 2011 11:00:46 +0100 Subject: [erlang-questions] Erlang User Conference 2011, Stockholm Message-ID: Hi Just to let you all know, the dates for the Erlang User Conference 2011 have been announced and talk submission is open! University: 31st October ? 2nd November Conference: 3rd November Tutorials: 4th November To Submit your talk or tutorial proposal go to: http://www.erlang-factory.com/conference/ErlangUserConference2011 Registration will be opening on the 15th August with an Early Bird Rate. We look forward to seeing you there! Thanks Jodie -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Thu Jul 7 13:31:09 2011 From: hd2010@REDACTED (H. Diedrich) Date: Thu, 07 Jul 2011 13:31:09 +0200 Subject: [erlang-questions] WTF? In-Reply-To: References: Message-ID: <4E1598FD.3060801@eonblast.com> On 6 July 2011 01:52, Mike Oxford wrote: > >> (Not to mention just plain annoying devs to the point >> they walk away from it, which is what kills more languages over time >> than anything else.) >> You are saying Erlang has the more annoying devs? Care to elaborate? Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Jul 7 13:51:51 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 7 Jul 2011 12:51:51 +0100 Subject: [erlang-questions] WTF? In-Reply-To: <4E1598FD.3060801@eonblast.com> References: <4E1598FD.3060801@eonblast.com> Message-ID: On 7 July 2011 12:31, H. Diedrich wrote: > On 6 July 2011 01:52, Mike Oxford wrote: > > (Not to mention just plain annoying devs to the point > they walk away from it, which is what kills more languages over time > than anything else.) > > > You are saying Erlang has the more annoying devs? > > Care to elaborate? > I think he meant that the tools annoy devs to the point they walk away before seeing the benefits. From crusso@REDACTED Thu Jul 7 14:08:16 2011 From: crusso@REDACTED (Claudio Russo) Date: Thu, 7 Jul 2011 12:08:16 +0000 Subject: [erlang-questions] CFP: PADL'12 - Practical Aspects of Declarative Languages 2012 Message-ID: <88D1F4047EA9A2468BCC6B743186880F58234CD3@DB3EX14MBXC309.europe.corp.microsoft.com> [Apologies if you receive multiple copies.] Call for Papers =============== 14th International Symposium on Practical Aspects of Declarative Languages (PADL 2012) http://research.microsoft.com/~crusso/padl12 Philadelphia, Pennsylvania, USA, January 23-24, 2012 Co-located with ACM POPL'12 Abstract/Paper submission deadline: September 10th/17th, 2011 Conference Description ====================== Declarative languages build on sound theoretical bases to provide attractive frameworks for application development. These languages have been successfully applied to many different real-world situations, ranging from data base management to active networks to software engineering to decision support systems. New developments in theory and implementation have opened up new application areas. At the same time, applications of declarative languages to novel problems raise numerous interesting research issues. Well-known questions include designing for scalability, language extensions for application deployment, and programming environments. Thus, applications drive the progress in the theory and implementation of declarative systems, and benefit from this progress as well. PADL is a forum for researchers and practitioners to present original work emphasizing novel applications and implementation techniques for all forms of declarative concepts, including, functional, logic, constraints, etc. Topics of interest include, but are not limited to: * Innovative applications of declarative languages * Declarative domain-specific languages and applications * Practical applications of theoretical results * New language developments and their impact on applications * Declarative languages and Software Engineering * Evaluation of implementation techniques on practical applications * Practical experiences and industrial applications * Novel uses of declarative languages in the classroom * Practical extensions such as constraint-based, probabilistic, and reactive languages. PADL'12 welcomes new ideas and approaches pertaining to applications and implementation of declarative languages. In this occasion PADL is co-located, as traditionally, with ACM POPL, which will be held immediately following PADL, January 25-27. The symposium will be held in Philadelphia, Pennsylvania, USA. Important Dates and Submission Guidelines ========================================= Abstract Submission: September 10, 2011 Paper Submission: September 17, 2011 Notification: October 22, 2011 Camera-ready: November 5, 2011 Symposium: January 23-24, 2011 Authors should submit an electronic copy of the full paper in PDF using the Springer LNCS format. The submission will be done through EasyChair conference system. If electronic submission is impossible, please contact the program chairs for information on how to submit hard copies. All submissions must be original work written in English. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted. PADL'12 will accept both technical and application papers: * Technical papers must describe original, previously unpublished research results. Technical papers must not exceed 15 pages in Springer LNCS format. * Application papers are a mechanism to present important practical applications of declarative languages that occur in industry or in areas of research other than Computer Science. Application papers will be published in the Springer-Verlag conference proceedings, and will be presented in a separate session. Application papers are expected to describe complex and/or real-world applications that rely on an innovative use of declarative languages. Application descriptions, engineering solutions and real-world experiences (both positive and negative) are solicited. The limit for application papers is 6 pages in Springer LNCS format. Program Committee ================= Marcello Balduccini, Intelligent Systems Department, Kodak Research Labs Edwin Brady, University of St Andrews, Scotland Henning Christiansen, Roskilde University, Denmark Agostino Dovier, University of Udine, Italy Matthew Flatt, University of Utah, USA Gopal Gupta, University of Texas at Dallas, USA John Hughes, Chalmers University of Technology, Sweden; Quviq AB Gabriele Keller, University of New South Wales, Australia Lunjin Lu, Oakland University, USA Marc Pouzet, ?cole normale sup?rieure, France Ricardo Rocha, University of Porto, Portugal Andreas Rossberg, Google Germany GmbH, Germany Claudio Russo, Microsoft Research Cambridge, UK (co-chair) Kostis Sagonas, Uppsala Univeristy, Sweden Satnam Singh, Microsoft Research Cambridge, UK Zoltan Somogyi, University of Melbourne, Australia Eijiro Sumii, Tohoku University, Japan Terrance Swift, Universidade Nova de Lisboa, Portugal; Johns Hopkins University, USA Andrew Tolmach, Portland State University, USA Jan Wielemaker, University of Amsterdam, The Netherlands Roland Yap, National University of Singapore, Republic of Singapore Kwangkeun Yi, Seoul National University, Korea Neng-Fa Zhou, Brooklyn College, City University of New York, USA (co-chair) Contacts ======== For additional information about papers and submissions, please contact the Program Chairs: Claudio Russo Microsoft Research Cambridge,UK Email: crusso microsoft com Neng-Fa Zhou Brooklyn College, The City University of New York, USA Email: zhou sci brooklyn cuny edu With the Cooperation of ======================= The Association for Logic Programming (ALP) ACM SIGPLAN =================================== From moxford@REDACTED Thu Jul 7 18:12:48 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 7 Jul 2011 09:12:48 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> Message-ID: On Thu, Jul 7, 2011 at 4:51 AM, Tim Watson wrote: > On 7 July 2011 12:31, H. Diedrich wrote: >> On 6 July 2011 01:52, Mike Oxford wrote: >> >> (Not to mention just plain annoying devs to the point >> they walk away from it, which is what kills more languages over time >> than anything else.) >> >> >> You are saying Erlang has the more annoying devs? >> >> Care to elaborate? >> > > I think he meant that the tools annoy devs to the point they walk away > before seeing the benefits. > Tim is correct; the erlang community is very good and I was referring to the toolset. "Not to mention just plain annoying (the) devs to the point (that) they walk away..." Sorry for the confusion. From karol.skocik@REDACTED Thu Jul 7 19:01:59 2011 From: karol.skocik@REDACTED (karol skocik) Date: Thu, 7 Jul 2011 19:01:59 +0200 Subject: [erlang-questions] mnesia race condition in add_table_copy Message-ID: Hi, I think I have found a race condition in mnesia:add_table_copy. I am trying to add table copy, when new node appears in cluster (or add table copy to another node, when the one having a copy fails), and the number of copies is less than some required count. The idea is simple, I spawn a new process on every node in cluster first, and in these processes I want to create a global transaction using global:trans with ID = {add_table_trans, table_name}. The first process which grabbed the transaction lock, checks if more table copies are required, and creates new copy on some node not having one, when needed. When the copy is created, this process exits, and another process on different node gets the transaction lock and tries to do the same. The problem here is, that the second process checks where are the copies using mnesia:table_info(table_name, disc_copies), and this list is sometimes incomplete, missing the very last node which got a table copy in the first process. It can be verified easily - in the second process: Copies1 = mnesia:table_info(table_name, disc_copies), timer:sleep(2000), Copies2 = mnesia:table_info(table_name, disc_copies). Then, mnesia:add_table_copy fails with {aborted,{already_exists,table_name,LastAddedNode}} Since the transaction lock ensures that no other process can add another table copy, I guess this is a race condition where new table copy node is not propagated to the schema on all nodes before the function mnesia:add_table_copy returns. Karol From moxford@REDACTED Thu Jul 7 20:09:29 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 7 Jul 2011 11:09:29 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> Message-ID: Closing the loop.... Background: 1) Rebar popped generating nodes. One this happens all regenerations fail until the Nodes are sanitized. (Issue filed https://issues.basho.com/show_bug.cgi?id=1132) 2) Node rebuilding failed due to backgrounded process 3) Rebar would no longer build in one particular directory ... the reltool spec was not generating correctly and would ONLY contain the .app file and none of the .beam entries. 4) Rebar would generate fine in other directories using the same source tree. Reboots did not fix it. Tree deletion did not fix it. The missing component? 1.5) See a problem, instantly make a backup "just in case" and name it "_backup". Why is this significant? Because at the same directory level, foo and foo_backup both appear to trip Rebar's pattern matching during a generate! Fix: rename "_backup" to "asdf__backup" to remove collision and all generations work correctly in all directories. Off to file a bug with Basho on it but something to be aware of in the meantime for Rebar users. Rock on. -mox On Thu, Jul 7, 2011 at 9:12 AM, Mike Oxford wrote: > On Thu, Jul 7, 2011 at 4:51 AM, Tim Watson wrote: >> On 7 July 2011 12:31, H. Diedrich wrote: >>> On 6 July 2011 01:52, Mike Oxford wrote: >>> >>> (Not to mention just plain annoying devs to the point >>> they walk away from it, which is what kills more languages over time >>> than anything else.) >>> >>> >>> You are saying Erlang has the more annoying devs? >>> >>> Care to elaborate? >>> >> >> I think he meant that the tools annoy devs to the point they walk away >> before seeing the benefits. >> > > Tim is correct; the erlang community is very good and I was referring > to the toolset. > > "Not to mention just plain annoying (the) devs to the point (that) > they walk away..." > > Sorry for the confusion. > From watson.timothy@REDACTED Thu Jul 7 20:16:25 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 7 Jul 2011 19:16:25 +0100 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> Message-ID: > The missing component? > 1.5) ?See a problem, instantly make a backup "just in case" and name > it "_backup". > > Why is this significant? > > Because at the same directory level, foo and foo_backup both appear to > trip Rebar's pattern matching during a generate! > > Fix: ?rename "_backup" to "asdf__backup" to remove > collision and all generations work correctly in all directories. > > Off to file a bug with Basho on it but something to be aware of in the > meantime for Rebar users. Thanks for letting the rest of us know in the meantime. From james@REDACTED Thu Jul 7 20:43:48 2011 From: james@REDACTED (James Aimonetti) Date: Thu, 7 Jul 2011 11:43:48 -0700 Subject: [erlang-questions] inet.erl socket_setopt type incomplete Message-ID: <4E15FE64.9010709@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 In R14B03, the type socket_setopt() in inet.erl does not allow for the convenience options 'binary' and 'list' (which internally get translated to {'mode', 'binary' | 'list'} which *is* in the type), and this causes Dialyzer to complain about inet:setopts(Socket, [{active, once}, binary]) (in my case). Setting the options list to [{active, once}, {mode, binary}] alleviates the Dialyzer warning but doesn't reflect what the function actually allows. - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOFf5kAAoJENc77s1OYoGgCAsH/3avCJlBdnJ0c78naoclMWre +AC9amjmTrVCkp6dg5mkCnMFsGpImCba171qMx3005pVS/e123px+6ek86cda9Gy rhIusElV8TR6r08QNzcbcdEHW89vbzYN6Gmc0AhnJX1gS6yBQ6vhn9tH3HxgaroZ N3VZCUAuyKaun2sWii7WfI0LdtxmWBxsZmUnpywPZhNOFm9A0hyY1rH533y+KIKJ kfleH3iO2E4tasE9U8eRNX2E9+fM00sXlVE84RUxVAonMcRTZyqBFxBa3nRStgii ShClwNH3CXnRiCgUgZhu6bW7Zd1Umq7Ru3gRRxcJTrTMJ2T8MfsD8EQ8/rgtVhw= =qgkH -----END PGP SIGNATURE----- From moxford@REDACTED Thu Jul 7 20:44:05 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 7 Jul 2011 11:44:05 -0700 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> Message-ID: On Thu, Jul 7, 2011 at 11:16 AM, Tim Watson wrote: > Thanks for letting the rest of us know in the meantime. No problemo. Rebar seems widely used and hopefully someone else will find this useful via the archives and save some time and headaches. :) https://issues.basho.com/show_bug.cgi?id=1134 It has the secondary effect of including .beam files for applications not actually used by your Node. For example, it would also distribute adsf__backup.ez -mox From erlang@REDACTED Thu Jul 7 21:29:23 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Jul 2011 21:29:23 +0200 Subject: [erlang-questions] web authentication Message-ID: Slightly off topic. But I want to make an erlang web site. 1) How does web authentication work? Let's assume something like: http://en.wikipedia.org/wiki/Digest_access_authentication This is easy to understand. What I don't understand is what happens if the session socket is closed. Handshaking tales place over an open socket and the client is authenticated - this is easy to understand. What happens if the socket is closed, and reopened in a subsequent request? Does the server set and receive a session cookie? Does the client remember and replay the authentication protocol? How does this work? 2) I want to make a web thing that requires the user to authenticate themself. Should I: a) Roll my own (some MD5 + cookies should do the job) b) Implement http://en.wikipedia.org/wiki/Digest_access_authentication c) Something else? Seems like for a real web site there is a lot of cruft involved preventing spammers, false-accounts, forgotten-passwords etc. can I get all of this for free by getting authentication credentials via goole/facebook or something? Is this what OpenID does? Finally is this entire authentication-user management-forgot my password built-in to any of the popular erlang web servers? Cheers /Joe From max.lapshin@REDACTED Thu Jul 7 21:35:01 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 7 Jul 2011 23:35:01 +0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: I don't advise you to use HTTP-based authorization. It is very unconvenient to do "logout" with it. Cookie based is more flexible to use and easier maintained. HTTP Header authorization and cookie authorization has nothing to do with connection and sockets, they are only per-request credentials. You should never believe that next request on the same socket is from the same client, because it may be HTTP proxy, that speaks with you. On Thu, Jul 7, 2011 at 11:29 PM, Joe Armstrong wrote: > Slightly off topic. But I want to make an erlang web site. > > 1) How does web authentication work? > > Let's assume something like: > So, there are basically two type of plain-text authorizations: using Auth: header and Cookie: header. Auth: header with 401 response will make your browser raise popup with login/password and Cookie: header is just a cookie. You may use it as you wish and for example put unique number of session. From tristan.sloughter@REDACTED Thu Jul 7 21:38:56 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 7 Jul 2011 14:38:56 -0500 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: My suggestion to simplify a lot of this: use HTTPS. And I agree with Max to not use HTTP-based authorization. I've implemented this along with password reset multiple times for Webmachine webapps... But I haven't yet separated it out for publishing to github. I hope to do this one day soon! Other frameworks that provide pieces (or all) of what you need are Nitrogen, ChicagoBoss and Zotonic. Tristan On Thu, Jul 7, 2011 at 2:35 PM, Max Lapshin wrote: > I don't advise you to use HTTP-based authorization. It is very > unconvenient to do "logout" with it. > Cookie based is more flexible to use and easier maintained. > > HTTP Header authorization and cookie authorization has nothing to do > with connection and sockets, > they are only per-request credentials. You should never believe that > next request on the same socket is from the same client, > because it may be HTTP proxy, that speaks with you. > > > > On Thu, Jul 7, 2011 at 11:29 PM, Joe Armstrong wrote: > > Slightly off topic. But I want to make an erlang web site. > > > > 1) How does web authentication work? > > > > Let's assume something like: > > > > So, there are basically two type of plain-text authorizations: using > Auth: header and Cookie: header. > > Auth: header with 401 response will make your browser raise popup with > login/password and Cookie: header is just a cookie. > You may use it as you wish and for example put unique number of session. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Thu Jul 7 21:39:33 2011 From: mihai@REDACTED (Mihai Balea) Date: Thu, 7 Jul 2011 15:39:33 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: <5C3C70E9-6A06-40F2-A09B-820B9DC64438@hates.ms> On Jul 7, 2011, at 3:29 PM, Joe Armstrong wrote: > What happens if the socket is closed, and reopened in a subsequent request? > Does the server set and receive a session cookie? Does the client remember and > replay the authentication protocol? If my understanding of things is correct, the client can use subsequent authenticated requests, by reusing the server supplied nonce, but issuing a different cnonce for each request. This can happen over one persistent http connection or many discrete connections. The server supplied nonce will expire after a time, and the auth protocol will have to be replayed. Mihai From andrew@REDACTED Thu Jul 7 21:49:12 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 7 Jul 2011 15:49:12 -0400 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> Message-ID: <20110707194912.GD30344@hijacked.us> I think this is reltool causing this weird behaviour, not riak. As a rule, using "../../" in your reltool config is probably going to bite you in the ass. Andrew From moxford@REDACTED Thu Jul 7 21:58:30 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 7 Jul 2011 12:58:30 -0700 Subject: [erlang-questions] WTF? In-Reply-To: <20110707194912.GD30344@hijacked.us> References: <4E1598FD.3060801@eonblast.com> <20110707194912.GD30344@hijacked.us> Message-ID: Using lib_dirs "../.." is, from what I understand, the canonical way of doing things and is in every tutorial I've ever seen. Given that environment build variables are not really an option, and not all developers on a project will have an explicit FQP to build from.... how would you suggest it be handled? Thanks! -mox On Thu, Jul 7, 2011 at 12:49 PM, Andrew Thompson wrote: > I think this is reltool causing this weird behaviour, not riak. As a > rule, using "../../" in your reltool config is probably going to bite > you in the ass. > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fred.hebert@REDACTED Thu Jul 7 22:04:58 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 7 Jul 2011 16:04:58 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: The digest authentication you see at this point is something that's part of the HTTP protocol. It is technically unrelated to your application (although your application can see it through HTTP values) and the client-side of it is implemented by the browser. I would consider it highly impractical for how little control it gives you (even in terms of visual design) and in general security. The browser being in charge of it means the session handling is annoying (no way to disconnect, the browser just resubmits the data all the time for you, proxies become tedious, etc.) If you're okay for a longer read, here's what I know, in condensed format. Others can feel free to correct me. ====== A QUICK GUIDE TO SAFER WEB AUTHENTICATION ====== Modern authentication schemes usually work a bit that way ---------- i. The user registers to the website on an HTTPS page (so that the password being sent isn't observable by third parties) ii. The password is stored in a database after being hashed by bcrypt or scrypt with a convenient, but slow work factor as to make brute-force (if the DB is compromised) absolutely impractical. These algorithms store their own difficulty factor, so that if you want to up the complexity, it can be done quite simply even if you have many versions in the DB. iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, some of the SHA functions too. MD5 and SHA hashing functions were made to be really fast and we want to avoid that. The reason is that it makes it easier to brute-force passwords if the table is compromised. Protect your users first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and give each of them a work factor. If you take 100 millisecond to check a password (something that happens once per session, so it's fine to be slow) compared to 10 microseconds, it becomes a real pain for crackers to do their thing. During that time, you can warn your users to change their passwords in other services. iv. bcrypt and scrypt handle salting for you, as mentioned before. If you decided not to use them, do remember to salt your passwords to avoid easier rainbow table attacks. Do NOT reapply a hash function many times over a previous hash results. Because all hash functions have collisions, this means that as you go, you increase the chances of collisions. Re-salt at each iteration. Again, if you use bcrypt or scrypt, this is handled for you and you do not have to worry. Please use bcrypt or scrypt. v. If that wasn't obvious, use the hashing function of bcrypt, not the encryption that can be decrypted. You do NOT want to use encryption, but hashing. The reason is simple. If your system is compromised, you have to assume that your keys are also compromised. Encryption is unsafe for passwords. Hashing is the king. That being said, the actual steps of authentication then become: 1. The login page is an HTTPS page so that the password that will be sent isn't observable by third parties 2. The user enters his own password, submits it 3. The server hashes it through the same hashing function used when first saving the password 4. Compare the two hash values. Note: you want this to be done in O(n) time so that it's impossible to know whether you get partial matches or not. This makes more sense for clear text passwords, but what the hell. 4.a if they're different, the authentication failed 4.b if they're the same, go to 5. 5. Create some kind of session value in another table or wherever you want. The session will have a key that you can choose randomly (UUIDv4 + userid to avoid duplicates although the chances are minimal, as an example). 6. Put the value in a cookie. You want that cookie to be set with the option HttpOnly to avoid having client-side javascript able to read it. Also give a reasonable timeout value to the cookie so the user has to re-authenticate from time to time. This means both server-time timeouts and client-side timeouts. When a user loads a page, check the session cookie you defined, and match its value to whatever you've got in your session table. If it matches, the user has an active session. If it doesn't, have the user authenticate again. Note that this scheme (and pretty much any other) is insecure against 3rd parties listening to the communications between you and the server. To solve this, move everything to HTTPS, although this is harder to optimise for performance. --- Further considerations: A. Put a limit on how many times someone can try to log in. You can have their account locked for a few minutes in between each time. This is to keep people from trying to brute force accounts from the outside. B. HTTP cookies alone are not sufficient to know whether someone is authenticated or not. A specific kind of attack, named Cross-Site Request Forgery (CSRF) works by having someone doing an HTTP request for you, from a different domain. In these cases, the browser will automatically send the cookies along with the request (it can also be a javascript form that's auto-submitted). I've used this in the past to have a site administrator (a co-worker) close his own account without him noticing it. Woops. This means we need an external value, not coming from HTTP headers to make sure the request is good. In simple cases, you can try to check the referrer and make sure it's coming from your domain. However, flash components or HTTPS connections do not always have these settings, and if you have any of these pages on your site (and you will, if you do HTTPS authentication), you'll have to look somewhere else. The safe way to do requires a few elements: 1. Never do changes with GET requests. GET is idempotent and free of side effects 2. Do your changes with POST. This will require a user to use either a) Javascript, b) curl, wget or some other tool or c) an HTML form to send your data. This restricts the window of opportunity of malevolent users, who can no longer just post a link or an image to do CSRF. 3. This is the most important point, use something called a token. This token is a special value that is added to whatever form you generate. When the user sends a POST query (or anything requiring server-side changes) to you, check that the token is there and that it is valid. The best tokens will have a one-time use, but they're fairly impractical when many different queries can be done without refreshing a page. My favourite way to handle these otherwise is to create different tokens for different namespaces. The admin panel will have its own token, the chat system its own token, the user settings will also have its own, etc. The tokens have to time out at some point to make sure that someone who steals them can not damage the account for too long. Pick between a few seconds to a few hours based on how long you expect your user to stay on a single page. 4. Ask for the user password when changing vital information, such as the e-mail attached to the account, the password itself, buying an item, etc. C. Never roll out your own encryption schemes or hash functions. Let mathematicians specialised in cryptography do their thing. ======== That should be about it for a quick guide. There are more complex issues to tackle for good authentication and security of user data, but as a basic intro, this should be okay. I hope this is helpful. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: > Slightly off topic. But I want to make an erlang web site. > > 1) How does web authentication work? > > Let's assume something like: > > http://en.wikipedia.org/wiki/Digest_access_authentication > > This is easy to understand. > > What I don't understand is what happens if the session socket is closed. > Handshaking tales place over an open socket and the client is > authenticated - this > is easy to understand. > > What happens if the socket is closed, and reopened in a subsequent request? > Does the server set and receive a session cookie? Does the client remember and > replay the authentication protocol? > > How does this work? > > 2) I want to make a web thing that requires the user to authenticate themself. > > Should I: > > a) Roll my own (some MD5 + cookies should do the job) > b) Implement http://en.wikipedia.org/wiki/Digest_access_authentication > c) Something else? > > Seems like for a real web site there is a lot of cruft involved > preventing spammers, > false-accounts, forgotten-passwords etc. can I get all of this for > free by getting > authentication credentials via goole/facebook or something? Is this > what OpenID does? > > Finally is this entire authentication-user management-forgot my > password built-in > to any of the popular erlang web servers? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andrew@REDACTED Thu Jul 7 22:07:40 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 7 Jul 2011 16:07:40 -0400 Subject: [erlang-questions] WTF? In-Reply-To: References: <4E1598FD.3060801@eonblast.com> <20110707194912.GD30344@hijacked.us> Message-ID: <20110707200740.GE30344@hijacked.us> On Thu, Jul 07, 2011 at 12:58:30PM -0700, Mike Oxford wrote: > Using lib_dirs "../.." is, from what I understand, the canonical way > of doing things and is in every tutorial I've ever seen. > > Given that environment build variables are not really an option, and > not all developers on a project will have an explicit FQP to build > from.... how would you suggest it be handled? > Look at how riak does it, it has a toplevel project that contains nothing but the release scripts, all the code is under deps. This avoids the problem of the parent directory contaminating the path when reltool looks for directories. Alternately I came up with an ugly hack with symlinks and a pre-compile hook: https://github.com/OpenACD/OpenACD/blob/master/pre_compile.sh#L17 Andrew From tristan.sloughter@REDACTED Thu Jul 7 22:08:35 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 7 Jul 2011 15:08:35 -0500 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: Really (in my opinion) everything should be in https, not just the login page, if you want to ensure that a users session is not stolen. Session stealing is very simple. A click of a button for a script kiddie on the same wireless network in the coffee shop with you. But the argument against that is "speed". I think the overhead https causes compared to the benefits and Internets speeds today make using https the way to go now. Tristan 2011/7/7 Fr?d?ric Trottier-H?bert > The digest authentication you see at this point is something that's part of > the HTTP protocol. It is technically unrelated to your application (although > your application can see it through HTTP values) and the client-side of it > is implemented by the browser. > > I would consider it highly impractical for how little control it gives you > (even in terms of visual design) and in general security. The browser being > in charge of it means the session handling is annoying (no way to > disconnect, the browser just resubmits the data all the time for you, > proxies become tedious, etc.) > > If you're okay for a longer read, here's what I know, in condensed format. > Others can feel free to correct me. > > ====== > A QUICK GUIDE TO SAFER WEB AUTHENTICATION > ====== > Modern authentication schemes usually work a bit that way > ---------- > > i. The user registers to the website on an HTTPS page (so that the password > being sent isn't observable by third parties) > > ii. The password is stored in a database after being hashed by bcrypt or > scrypt with a convenient, but slow work factor as to make brute-force (if > the DB is compromised) absolutely impractical. These algorithms store their > own difficulty factor, so that if you want to up the complexity, it can be > done quite simply even if you have many versions in the DB. > > iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, > some of the SHA functions too. MD5 and SHA hashing functions were made to be > really fast and we want to avoid that. The reason is that it makes it easier > to brute-force passwords if the table is compromised. Protect your users > first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and > give each of them a work factor. If you take 100 millisecond to check a > password (something that happens once per session, so it's fine to be slow) > compared to 10 microseconds, it becomes a real pain for crackers to do their > thing. During that time, you can warn your users to change their passwords > in other services. > > iv. bcrypt and scrypt handle salting for you, as mentioned before. If you > decided not to use them, do remember to salt your passwords to avoid easier > rainbow table attacks. Do NOT reapply a hash function many times over a > previous hash results. Because all hash functions have collisions, this > means that as you go, you increase the chances of collisions. Re-salt at > each iteration. Again, if you use bcrypt or scrypt, this is handled for you > and you do not have to worry. Please use bcrypt or scrypt. > > v. If that wasn't obvious, use the hashing function of bcrypt, not the > encryption that can be decrypted. You do NOT want to use encryption, but > hashing. The reason is simple. If your system is compromised, you have to > assume that your keys are also compromised. Encryption is unsafe for > passwords. Hashing is the king. > > That being said, the actual steps of authentication then become: > > 1. The login page is an HTTPS page so that the password that will be sent > isn't observable by third parties > 2. The user enters his own password, submits it > 3. The server hashes it through the same hashing function used when first > saving the password > 4. Compare the two hash values. Note: you want this to be done in O(n) time > so that it's impossible to know whether you get partial matches or not. This > makes more sense for clear text passwords, but what the hell. > 4.a if they're different, the authentication failed > 4.b if they're the same, go to 5. > 5. Create some kind of session value in another table or wherever you want. > The session will have a key that you can choose randomly (UUIDv4 + userid to > avoid duplicates although the chances are minimal, as an example). > 6. Put the value in a cookie. You want that cookie to be set with the > option HttpOnly to avoid having client-side javascript able to read it. Also > give a reasonable timeout value to the cookie so the user has to > re-authenticate from time to time. This means both server-time timeouts and > client-side timeouts. > > When a user loads a page, check the session cookie you defined, and match > its value to whatever you've got in your session table. If it matches, the > user has an active session. If it doesn't, have the user authenticate again. > > Note that this scheme (and pretty much any other) is insecure against 3rd > parties listening to the communications between you and the server. To solve > this, move everything to HTTPS, although this is harder to optimise for > performance. > > --- > Further considerations: > > A. Put a limit on how many times someone can try to log in. You can have > their account locked for a few minutes in between each time. This is to keep > people from trying to brute force accounts from the outside. > > B. HTTP cookies alone are not sufficient to know whether someone is > authenticated or not. A specific kind of attack, named Cross-Site Request > Forgery (CSRF) works by having someone doing an HTTP request for you, from a > different domain. In these cases, the browser will automatically send the > cookies along with the request (it can also be a javascript form that's > auto-submitted). I've used this in the past to have a site administrator (a > co-worker) close his own account without him noticing it. Woops. > > This means we need an external value, not coming from HTTP headers to make > sure the request is good. > > In simple cases, you can try to check the referrer and make sure it's > coming from your domain. However, flash components or HTTPS connections do > not always have these settings, and if you have any of these pages on your > site (and you will, if you do HTTPS authentication), you'll have to look > somewhere else. > > The safe way to do requires a few elements: > > 1. Never do changes with GET requests. GET is idempotent and free of side > effects > > 2. Do your changes with POST. This will require a user to use either a) > Javascript, b) curl, wget or some other tool or c) an HTML form to send your > data. This restricts the window of opportunity of malevolent users, who can > no longer just post a link or an image to do CSRF. > > 3. This is the most important point, use something called a token. This > token is a special value that is added to whatever form you generate. When > the user sends a POST query (or anything requiring server-side changes) to > you, check that the token is there and that it is valid. The best tokens > will have a one-time use, but they're fairly impractical when many different > queries can be done without refreshing a page. My favourite way to handle > these otherwise is to create different tokens for different namespaces. The > admin panel will have its own token, the chat system its own token, the user > settings will also have its own, etc. > > The tokens have to time out at some point to make sure that someone who > steals them can not damage the account for too long. Pick between a few > seconds to a few hours based on how long you expect your user to stay on a > single page. > > 4. Ask for the user password when changing vital information, such as the > e-mail attached to the account, the password itself, buying an item, etc. > > C. Never roll out your own encryption schemes or hash functions. Let > mathematicians specialised in cryptography do their thing. > > ======== > > That should be about it for a quick guide. There are more complex issues to > tackle for good authentication and security of user data, but as a basic > intro, this should be okay. I hope this is helpful. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: > > > Slightly off topic. But I want to make an erlang web site. > > > > 1) How does web authentication work? > > > > Let's assume something like: > > > > http://en.wikipedia.org/wiki/Digest_access_authentication > > > > This is easy to understand. > > > > What I don't understand is what happens if the session socket is closed. > > Handshaking tales place over an open socket and the client is > > authenticated - this > > is easy to understand. > > > > What happens if the socket is closed, and reopened in a subsequent > request? > > Does the server set and receive a session cookie? Does the client > remember and > > replay the authentication protocol? > > > > How does this work? > > > > 2) I want to make a web thing that requires the user to authenticate > themself. > > > > Should I: > > > > a) Roll my own (some MD5 + cookies should do the job) > > b) Implement > http://en.wikipedia.org/wiki/Digest_access_authentication > > c) Something else? > > > > Seems like for a real web site there is a lot of cruft involved > > preventing spammers, > > false-accounts, forgotten-passwords etc. can I get all of this for > > free by getting > > authentication credentials via goole/facebook or something? Is this > > what OpenID does? > > > > Finally is this entire authentication-user management-forgot my > > password built-in > > to any of the popular erlang web servers? > > > > Cheers > > > > /Joe > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred.hebert@REDACTED Thu Jul 7 22:10:37 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 7 Jul 2011 16:10:37 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: <5E989ACE-2D46-44D4-98A1-5A63A625516A@erlang-solutions.com> True, but HTTPS has performance impacts that sometimes people find unacceptable. If you read down the guide you'll see that I do mention HTTPS by default, but a strict minimum should be the login page. You want to protect the user password more than the user's data on your site, for the simple reason that the password can be used in other places and the very risky operations should always ask for a password. HTTPS everywhere is the safest way to communicate with the server, yes. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-07, at 16:08 PM, Tristan Sloughter wrote: > Really (in my opinion) everything should be in https, not just the login page, if you want to ensure that a users session is not stolen. Session stealing is very simple. A click of a button for a script kiddie on the same wireless network in the coffee shop with you. But the argument against that is "speed". I think the overhead https causes compared to the benefits and Internets speeds today make using https the way to go now. > > Tristan > > 2011/7/7 Fr?d?ric Trottier-H?bert > The digest authentication you see at this point is something that's part of the HTTP protocol. It is technically unrelated to your application (although your application can see it through HTTP values) and the client-side of it is implemented by the browser. > > I would consider it highly impractical for how little control it gives you (even in terms of visual design) and in general security. The browser being in charge of it means the session handling is annoying (no way to disconnect, the browser just resubmits the data all the time for you, proxies become tedious, etc.) > > If you're okay for a longer read, here's what I know, in condensed format. Others can feel free to correct me. > > ====== > A QUICK GUIDE TO SAFER WEB AUTHENTICATION > ====== > Modern authentication schemes usually work a bit that way > ---------- > > i. The user registers to the website on an HTTPS page (so that the password being sent isn't observable by third parties) > > ii. The password is stored in a database after being hashed by bcrypt or scrypt with a convenient, but slow work factor as to make brute-force (if the DB is compromised) absolutely impractical. These algorithms store their own difficulty factor, so that if you want to up the complexity, it can be done quite simply even if you have many versions in the DB. > > iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, some of the SHA functions too. MD5 and SHA hashing functions were made to be really fast and we want to avoid that. The reason is that it makes it easier to brute-force passwords if the table is compromised. Protect your users first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and give each of them a work factor. If you take 100 millisecond to check a password (something that happens once per session, so it's fine to be slow) compared to 10 microseconds, it becomes a real pain for crackers to do their thing. During that time, you can warn your users to change their passwords in other services. > > iv. bcrypt and scrypt handle salting for you, as mentioned before. If you decided not to use them, do remember to salt your passwords to avoid easier rainbow table attacks. Do NOT reapply a hash function many times over a previous hash results. Because all hash functions have collisions, this means that as you go, you increase the chances of collisions. Re-salt at each iteration. Again, if you use bcrypt or scrypt, this is handled for you and you do not have to worry. Please use bcrypt or scrypt. > > v. If that wasn't obvious, use the hashing function of bcrypt, not the encryption that can be decrypted. You do NOT want to use encryption, but hashing. The reason is simple. If your system is compromised, you have to assume that your keys are also compromised. Encryption is unsafe for passwords. Hashing is the king. > > That being said, the actual steps of authentication then become: > > 1. The login page is an HTTPS page so that the password that will be sent isn't observable by third parties > 2. The user enters his own password, submits it > 3. The server hashes it through the same hashing function used when first saving the password > 4. Compare the two hash values. Note: you want this to be done in O(n) time so that it's impossible to know whether you get partial matches or not. This makes more sense for clear text passwords, but what the hell. > 4.a if they're different, the authentication failed > 4.b if they're the same, go to 5. > 5. Create some kind of session value in another table or wherever you want. The session will have a key that you can choose randomly (UUIDv4 + userid to avoid duplicates although the chances are minimal, as an example). > 6. Put the value in a cookie. You want that cookie to be set with the option HttpOnly to avoid having client-side javascript able to read it. Also give a reasonable timeout value to the cookie so the user has to re-authenticate from time to time. This means both server-time timeouts and client-side timeouts. > > When a user loads a page, check the session cookie you defined, and match its value to whatever you've got in your session table. If it matches, the user has an active session. If it doesn't, have the user authenticate again. > > Note that this scheme (and pretty much any other) is insecure against 3rd parties listening to the communications between you and the server. To solve this, move everything to HTTPS, although this is harder to optimise for performance. > > --- > Further considerations: > > A. Put a limit on how many times someone can try to log in. You can have their account locked for a few minutes in between each time. This is to keep people from trying to brute force accounts from the outside. > > B. HTTP cookies alone are not sufficient to know whether someone is authenticated or not. A specific kind of attack, named Cross-Site Request Forgery (CSRF) works by having someone doing an HTTP request for you, from a different domain. In these cases, the browser will automatically send the cookies along with the request (it can also be a javascript form that's auto-submitted). I've used this in the past to have a site administrator (a co-worker) close his own account without him noticing it. Woops. > > This means we need an external value, not coming from HTTP headers to make sure the request is good. > > In simple cases, you can try to check the referrer and make sure it's coming from your domain. However, flash components or HTTPS connections do not always have these settings, and if you have any of these pages on your site (and you will, if you do HTTPS authentication), you'll have to look somewhere else. > > The safe way to do requires a few elements: > > 1. Never do changes with GET requests. GET is idempotent and free of side effects > > 2. Do your changes with POST. This will require a user to use either a) Javascript, b) curl, wget or some other tool or c) an HTML form to send your data. This restricts the window of opportunity of malevolent users, who can no longer just post a link or an image to do CSRF. > > 3. This is the most important point, use something called a token. This token is a special value that is added to whatever form you generate. When the user sends a POST query (or anything requiring server-side changes) to you, check that the token is there and that it is valid. The best tokens will have a one-time use, but they're fairly impractical when many different queries can be done without refreshing a page. My favourite way to handle these otherwise is to create different tokens for different namespaces. The admin panel will have its own token, the chat system its own token, the user settings will also have its own, etc. > > The tokens have to time out at some point to make sure that someone who steals them can not damage the account for too long. Pick between a few seconds to a few hours based on how long you expect your user to stay on a single page. > > 4. Ask for the user password when changing vital information, such as the e-mail attached to the account, the password itself, buying an item, etc. > > C. Never roll out your own encryption schemes or hash functions. Let mathematicians specialised in cryptography do their thing. > > ======== > > That should be about it for a quick guide. There are more complex issues to tackle for good authentication and security of user data, but as a basic intro, this should be okay. I hope this is helpful. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: > > > Slightly off topic. But I want to make an erlang web site. > > > > 1) How does web authentication work? > > > > Let's assume something like: > > > > http://en.wikipedia.org/wiki/Digest_access_authentication > > > > This is easy to understand. > > > > What I don't understand is what happens if the session socket is closed. > > Handshaking tales place over an open socket and the client is > > authenticated - this > > is easy to understand. > > > > What happens if the socket is closed, and reopened in a subsequent request? > > Does the server set and receive a session cookie? Does the client remember and > > replay the authentication protocol? > > > > How does this work? > > > > 2) I want to make a web thing that requires the user to authenticate themself. > > > > Should I: > > > > a) Roll my own (some MD5 + cookies should do the job) > > b) Implement http://en.wikipedia.org/wiki/Digest_access_authentication > > c) Something else? > > > > Seems like for a real web site there is a lot of cruft involved > > preventing spammers, > > false-accounts, forgotten-passwords etc. can I get all of this for > > free by getting > > authentication credentials via goole/facebook or something? Is this > > what OpenID does? > > > > Finally is this entire authentication-user management-forgot my > > password built-in > > to any of the popular erlang web servers? > > > > Cheers > > > > /Joe > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jul 7 23:12:01 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 07 Jul 2011 22:12:01 +0100 Subject: [erlang-questions] No cmd window Message-ID: <4E162121.9080306@bobcowdery.plus.com> Hi Is there any way to have an erlang app run without a cmd window under Windows. It's a GUI app and the command window just gets in the way. It seems that erlang always runs in a cmd window regardless of how its started. I've tried -detached and that just exits the application immediately, -noshell stops it taking input but still starts a cmd window. I even tried starting it from Python using the windows .pyw extension which runs without a cmd window but still erl opens one. Regards Bob From octopusfluff@REDACTED Thu Jul 7 23:36:18 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Thu, 7 Jul 2011 14:36:18 -0700 Subject: [erlang-questions] No cmd window In-Reply-To: <4E162121.9080306@bobcowdery.plus.com> References: <4E162121.9080306@bobcowdery.plus.com> Message-ID: On Thu, Jul 7, 2011 at 2:12 PM, Bob Cowdery wrote: > Is there any way to have an erlang app run without a cmd window under > Windows. It's a GUI app and the command window just gets in the way. It > seems that erlang always runs in a cmd window regardless of how its > started. I've tried -detached and that just exits the application > immediately, -noshell stops it taking input but still starts a cmd > window. I even tried starting it from Python using the windows .pyw > extension which runs without a cmd window but still erl opens one. At this time it doesn't seem to be possible. From dangud@REDACTED Fri Jul 8 00:19:31 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 8 Jul 2011 00:19:31 +0200 Subject: [erlang-questions] No cmd window In-Reply-To: References: <4E162121.9080306@bobcowdery.plus.com> Message-ID: werl -detached -run wx demo works fine for me /Dan On Thu, Jul 7, 2011 at 11:36 PM, Amy Lear wrote: > On Thu, Jul 7, 2011 at 2:12 PM, Bob Cowdery > wrote: > > Is there any way to have an erlang app run without a cmd window under > > Windows. It's a GUI app and the command window just gets in the way. It > > seems that erlang always runs in a cmd window regardless of how its > > started. I've tried -detached and that just exits the application > > immediately, -noshell stops it taking input but still starts a cmd > > window. I even tried starting it from Python using the windows .pyw > > extension which runs without a cmd window but still erl opens one. > > At this time it doesn't seem to be possible. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Fri Jul 8 01:49:53 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Thu, 7 Jul 2011 16:49:53 -0700 Subject: [erlang-questions] No cmd window In-Reply-To: References: <4E162121.9080306@bobcowdery.plus.com> Message-ID: On Thu, Jul 7, 2011 at 3:19 PM, Dan Gudmundsson wrote: > werl -detached -run wx demo > works fine for me Well that sure is handy. I learned two things here. Thanks! From banibrata.dutta@REDACTED Fri Jul 8 05:25:45 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 8 Jul 2011 08:55:45 +0530 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: Thanks for the amazing, simple-worded, web-authentication run down. All the other comments so far as as great. Gets my "thread of the month" award ("movie of the month" award IMHO, goes to the Erlang web-server perf comparison movie). Am willing to spend some money on buying a mini e/Book (Wikibooks?) or the likes if this kind of info is turned into a "Learn You Some Web-Authentication"... or slightly broader "Learn You Some Practical & Professional Web-Development" :-). And if it uses Erlang as the lang of choice for examples, all the more better. [subtle hint ;-)] 2011/7/8 Fr?d?ric Trottier-H?bert > The digest authentication you see at this point is something that's part of > the HTTP protocol. It is technically unrelated to your application (although > your application can see it through HTTP values) and the client-side of it > is implemented by the browser. > > I would consider it highly impractical for how little control it gives you > (even in terms of visual design) and in general security. The browser being > in charge of it means the session handling is annoying (no way to > disconnect, the browser just resubmits the data all the time for you, > proxies become tedious, etc.) > > If you're okay for a longer read, here's what I know, in condensed format. > Others can feel free to correct me. > > ====== > A QUICK GUIDE TO SAFER WEB AUTHENTICATION > ====== > Modern authentication schemes usually work a bit that way > ---------- > > i. The user registers to the website on an HTTPS page (so that the password > being sent isn't observable by third parties) > > ii. The password is stored in a database after being hashed by bcrypt or > scrypt with a convenient, but slow work factor as to make brute-force (if > the DB is compromised) absolutely impractical. These algorithms store their > own difficulty factor, so that if you want to up the complexity, it can be > done quite simply even if you have many versions in the DB. > > iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, > some of the SHA functions too. MD5 and SHA hashing functions were made to be > really fast and we want to avoid that. The reason is that it makes it easier > to brute-force passwords if the table is compromised. Protect your users > first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and > give each of them a work factor. If you take 100 millisecond to check a > password (something that happens once per session, so it's fine to be slow) > compared to 10 microseconds, it becomes a real pain for crackers to do their > thing. During that time, you can warn your users to change their passwords > in other services. > > iv. bcrypt and scrypt handle salting for you, as mentioned before. If you > decided not to use them, do remember to salt your passwords to avoid easier > rainbow table attacks. Do NOT reapply a hash function many times over a > previous hash results. Because all hash functions have collisions, this > means that as you go, you increase the chances of collisions. Re-salt at > each iteration. Again, if you use bcrypt or scrypt, this is handled for you > and you do not have to worry. Please use bcrypt or scrypt. > > v. If that wasn't obvious, use the hashing function of bcrypt, not the > encryption that can be decrypted. You do NOT want to use encryption, but > hashing. The reason is simple. If your system is compromised, you have to > assume that your keys are also compromised. Encryption is unsafe for > passwords. Hashing is the king. > > That being said, the actual steps of authentication then become: > > 1. The login page is an HTTPS page so that the password that will be sent > isn't observable by third parties > 2. The user enters his own password, submits it > 3. The server hashes it through the same hashing function used when first > saving the password > 4. Compare the two hash values. Note: you want this to be done in O(n) time > so that it's impossible to know whether you get partial matches or not. This > makes more sense for clear text passwords, but what the hell. > 4.a if they're different, the authentication failed > 4.b if they're the same, go to 5. > 5. Create some kind of session value in another table or wherever you want. > The session will have a key that you can choose randomly (UUIDv4 + userid to > avoid duplicates although the chances are minimal, as an example). > 6. Put the value in a cookie. You want that cookie to be set with the > option HttpOnly to avoid having client-side javascript able to read it. Also > give a reasonable timeout value to the cookie so the user has to > re-authenticate from time to time. This means both server-time timeouts and > client-side timeouts. > > When a user loads a page, check the session cookie you defined, and match > its value to whatever you've got in your session table. If it matches, the > user has an active session. If it doesn't, have the user authenticate again. > > Note that this scheme (and pretty much any other) is insecure against 3rd > parties listening to the communications between you and the server. To solve > this, move everything to HTTPS, although this is harder to optimise for > performance. > > --- > Further considerations: > > A. Put a limit on how many times someone can try to log in. You can have > their account locked for a few minutes in between each time. This is to keep > people from trying to brute force accounts from the outside. > > B. HTTP cookies alone are not sufficient to know whether someone is > authenticated or not. A specific kind of attack, named Cross-Site Request > Forgery (CSRF) works by having someone doing an HTTP request for you, from a > different domain. In these cases, the browser will automatically send the > cookies along with the request (it can also be a javascript form that's > auto-submitted). I've used this in the past to have a site administrator (a > co-worker) close his own account without him noticing it. Woops. > > This means we need an external value, not coming from HTTP headers to make > sure the request is good. > > In simple cases, you can try to check the referrer and make sure it's > coming from your domain. However, flash components or HTTPS connections do > not always have these settings, and if you have any of these pages on your > site (and you will, if you do HTTPS authentication), you'll have to look > somewhere else. > > The safe way to do requires a few elements: > > 1. Never do changes with GET requests. GET is idempotent and free of side > effects > > 2. Do your changes with POST. This will require a user to use either a) > Javascript, b) curl, wget or some other tool or c) an HTML form to send your > data. This restricts the window of opportunity of malevolent users, who can > no longer just post a link or an image to do CSRF. > > 3. This is the most important point, use something called a token. This > token is a special value that is added to whatever form you generate. When > the user sends a POST query (or anything requiring server-side changes) to > you, check that the token is there and that it is valid. The best tokens > will have a one-time use, but they're fairly impractical when many different > queries can be done without refreshing a page. My favourite way to handle > these otherwise is to create different tokens for different namespaces. The > admin panel will have its own token, the chat system its own token, the user > settings will also have its own, etc. > > The tokens have to time out at some point to make sure that someone who > steals them can not damage the account for too long. Pick between a few > seconds to a few hours based on how long you expect your user to stay on a > single page. > > 4. Ask for the user password when changing vital information, such as the > e-mail attached to the account, the password itself, buying an item, etc. > > C. Never roll out your own encryption schemes or hash functions. Let > mathematicians specialised in cryptography do their thing. > > ======== > > That should be about it for a quick guide. There are more complex issues to > tackle for good authentication and security of user data, but as a basic > intro, this should be okay. I hope this is helpful. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: > > > Slightly off topic. But I want to make an erlang web site. > > > > 1) How does web authentication work? > > > > Let's assume something like: > > > > http://en.wikipedia.org/wiki/Digest_access_authentication > > > > This is easy to understand. > > > > What I don't understand is what happens if the session socket is closed. > > Handshaking tales place over an open socket and the client is > > authenticated - this > > is easy to understand. > > > > What happens if the socket is closed, and reopened in a subsequent > request? > > Does the server set and receive a session cookie? Does the client > remember and > > replay the authentication protocol? > > > > How does this work? > > > > 2) I want to make a web thing that requires the user to authenticate > themself. > > > > Should I: > > > > a) Roll my own (some MD5 + cookies should do the job) > > b) Implement > http://en.wikipedia.org/wiki/Digest_access_authentication > > c) Something else? > > > > Seems like for a real web site there is a lot of cruft involved > > preventing spammers, > > false-accounts, forgotten-passwords etc. can I get all of this for > > free by getting > > authentication credentials via goole/facebook or something? Is this > > what OpenID does? > > > > Finally is this entire authentication-user management-forgot my > > password built-in > > to any of the popular erlang web servers? > > > > Cheers > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Jul 8 06:26:40 2011 From: gumm@REDACTED (Jesse Gumm) Date: Thu, 7 Jul 2011 23:26:40 -0500 Subject: [erlang-questions] Erlang Web Servers challenge In-Reply-To: <86989A08-AEB3-44F7-89EB-EE0C78F9AF87@llaisdy.com> References: <00f001cc3bf6$e94f3e60$bbedbb20$@com> <86989A08-AEB3-44F7-89EB-EE0C78F9AF87@llaisdy.com> Message-ID: Well played Garrett! That's too funny. On Thu, Jul 7, 2011 at 2:53 AM, Ivan Uemlianin wrote: > +2! > > And the synthetic voices aren't bad either. xtranormal looks pretty cool. > > Ivan > > Sent from iPhone. > > On 7 Jul 2011, at 07:34, Joe Armstrong wrote: > >> On Thu, Jul 7, 2011 at 12:35 AM, Garrett Smith wrote: >>> Interesting thread. Reminds me of a conversation I overhead, in >>> different time and place: >>> >>> http://www.youtube.com/watch?v=xEJ1n13soWU >>> >>> Very slightly NSFW language. >> >> Wonderfull - any ideas how this movie was made? >> >> /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Sigma Star Systems 414.940.4866 gumm@REDACTED http://www.sigma-star.com From andrey.pampukha@REDACTED Fri Jul 8 09:50:50 2011 From: andrey.pampukha@REDACTED (Andrey Pampukha) Date: Fri, 8 Jul 2011 11:50:50 +0400 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: <1E2B4AB6-1D4E-43CB-9C6D-55C7688B1269@verivue.com> References: <1E2B4AB6-1D4E-43CB-9C6D-55C7688B1269@verivue.com> Message-ID: Try out this link: http://erlang-factory.com/conference/London2011/talks They all are here :) 2011/7/5 Evans, Matthew > Yes, I was looking for these slides too > > Sent from my iPhone > > On Jun 26, 2011, at 2:20 PM, "Paul Fisher" wrote: > > > Kenneth, > > > > Can you publish your slides from your Erlang Factory talk on the OTP > Roadmap? I apologize if you have already done so, I was looking on the > Erlang Factory site and a link was not available. > > > > > > -- > > paul > > > > director, platform services > > alertlogic, inc. > > 713-484-8383 x2314 > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Jul 8 10:22:53 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 08 Jul 2011 09:22:53 +0100 Subject: [erlang-questions] No cmd window In-Reply-To: References: <4E162121.9080306@bobcowdery.plus.com> Message-ID: <4E16BE5D.3080301@bobcowdery.plus.com> werl -detached gives me the rather odd error message: Its clearly the wrong path. Werl on its own works fine. I'm on Windows 7. Bob On 07/07/2011 23:19, Dan Gudmundsson wrote: > werl -detached -run wx demo > > works fine for me > > /Dan > > On Thu, Jul 7, 2011 at 11:36 PM, Amy Lear > wrote: > > On Thu, Jul 7, 2011 at 2:12 PM, Bob Cowdery > > wrote: > > Is there any way to have an erlang app run without a cmd window > under > > Windows. It's a GUI app and the command window just gets in the > way. It > > seems that erlang always runs in a cmd window regardless of how its > > started. I've tried -detached and that just exits the application > > immediately, -noshell stops it taking input but still starts a cmd > > window. I even tried starting it from Python using the windows .pyw > > extension which runs without a cmd window but still erl opens one. > > At this time it doesn't seem to be possible. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: moz-screenshot-1.png Type: image/png Size: 28154 bytes Desc: not available URL: From karol.skocik@REDACTED Fri Jul 8 10:30:41 2011 From: karol.skocik@REDACTED (karol skocik) Date: Fri, 8 Jul 2011 10:30:41 +0200 Subject: [erlang-questions] mnesia race condition in add_table_copy In-Reply-To: References: Message-ID: In addition to this, the obvious question: what should I do to ensure that schema change is propagated after mnesia:add_table_copy? Or alternatively, what should I call **before** mnesia:table_info(Table, disc_copies) to get all nodes where the copy resides? Thanks for any suggestions, Karol On Thu, Jul 7, 2011 at 7:01 PM, karol skocik wrote: > Hi, > ?I think I have found a race condition in mnesia:add_table_copy. > I am trying to add table copy, when new node appears in cluster (or > add table copy to another node, when the one having a copy fails), and > the number of copies is less than some required count. > > The idea is simple, I spawn a new process on every node in cluster > first, and in these processes I want to create a global transaction > using global:trans with ID = {add_table_trans, table_name}. > The first process which grabbed the transaction lock, checks if more > table copies are required, and creates new copy on some node not > having one, when needed. > When the copy is created, this process exits, and another process on > different node gets the transaction lock and tries to do the same. > > The problem here is, that the second process checks where are the > copies using mnesia:table_info(table_name, disc_copies), and this list > is sometimes incomplete, missing the very last node which got a table > copy in the first process. > It can be verified easily - in the second process: > Copies1 = mnesia:table_info(table_name, disc_copies), > timer:sleep(2000), > Copies2 = mnesia:table_info(table_name, disc_copies). > > Then, mnesia:add_table_copy fails with > {aborted,{already_exists,table_name,LastAddedNode}} > > Since the transaction lock ensures that no other process can add > another table copy, I guess this is a race condition where new table > copy node is not propagated to the schema on all nodes before the > function mnesia:add_table_copy returns. > > Karol > From dangud@REDACTED Fri Jul 8 10:38:03 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 8 Jul 2011 10:38:03 +0200 Subject: [erlang-questions] mnesia race condition in add_table_copy In-Reply-To: References: Message-ID: Can you send me (privately) a small test program that shows the problem. /Dan On Fri, Jul 8, 2011 at 10:30 AM, karol skocik wrote: > In addition to this, the obvious question: > what should I do to ensure that schema change is propagated after > mnesia:add_table_copy? > > Or alternatively, what should I call **before** > mnesia:table_info(Table, disc_copies) to get all nodes where the copy > resides? > > Thanks for any suggestions, > Karol > > On Thu, Jul 7, 2011 at 7:01 PM, karol skocik > wrote: > > Hi, > > I think I have found a race condition in mnesia:add_table_copy. > > I am trying to add table copy, when new node appears in cluster (or > > add table copy to another node, when the one having a copy fails), and > > the number of copies is less than some required count. > > > > The idea is simple, I spawn a new process on every node in cluster > > first, and in these processes I want to create a global transaction > > using global:trans with ID = {add_table_trans, table_name}. > > The first process which grabbed the transaction lock, checks if more > > table copies are required, and creates new copy on some node not > > having one, when needed. > > When the copy is created, this process exits, and another process on > > different node gets the transaction lock and tries to do the same. > > > > The problem here is, that the second process checks where are the > > copies using mnesia:table_info(table_name, disc_copies), and this list > > is sometimes incomplete, missing the very last node which got a table > > copy in the first process. > > It can be verified easily - in the second process: > > Copies1 = mnesia:table_info(table_name, disc_copies), > > timer:sleep(2000), > > Copies2 = mnesia:table_info(table_name, disc_copies). > > > > Then, mnesia:add_table_copy fails with > > {aborted,{already_exists,table_name,LastAddedNode}} > > > > Since the transaction lock ensures that no other process can add > > another table copy, I guess this is a race condition where new table > > copy node is not propagated to the schema on all nodes before the > > function mnesia:add_table_copy returns. > > > > Karol > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From karol.skocik@REDACTED Fri Jul 8 11:25:10 2011 From: karol.skocik@REDACTED (karol skocik) Date: Fri, 8 Jul 2011 11:25:10 +0200 Subject: [erlang-questions] mnesia race condition in add_table_copy In-Reply-To: References: Message-ID: I will do that later this evening, unless I find a fix for it. Unfortunately to reproduce it, it's "non-small" amount of setup code for cluster/mnesia startup I will need to separate. Thanks, Karol On Fri, Jul 8, 2011 at 10:38 AM, Dan Gudmundsson wrote: > Can you send me (privately) a small test program that shows the problem. > > /Dan > > On Fri, Jul 8, 2011 at 10:30 AM, karol skocik > wrote: >> >> In addition to this, the obvious question: >> what should I do to ensure that schema change is propagated after >> mnesia:add_table_copy? >> >> Or alternatively, what should I call **before** >> mnesia:table_info(Table, disc_copies) to get all nodes where the copy >> resides? >> >> Thanks for any suggestions, >> Karol >> >> On Thu, Jul 7, 2011 at 7:01 PM, karol skocik >> wrote: >> > Hi, >> > ?I think I have found a race condition in mnesia:add_table_copy. >> > I am trying to add table copy, when new node appears in cluster (or >> > add table copy to another node, when the one having a copy fails), and >> > the number of copies is less than some required count. >> > >> > The idea is simple, I spawn a new process on every node in cluster >> > first, and in these processes I want to create a global transaction >> > using global:trans with ID = {add_table_trans, table_name}. >> > The first process which grabbed the transaction lock, checks if more >> > table copies are required, and creates new copy on some node not >> > having one, when needed. >> > When the copy is created, this process exits, and another process on >> > different node gets the transaction lock and tries to do the same. >> > >> > The problem here is, that the second process checks where are the >> > copies using mnesia:table_info(table_name, disc_copies), and this list >> > is sometimes incomplete, missing the very last node which got a table >> > copy in the first process. >> > It can be verified easily - in the second process: >> > Copies1 = mnesia:table_info(table_name, disc_copies), >> > timer:sleep(2000), >> > Copies2 = mnesia:table_info(table_name, disc_copies). >> > >> > Then, mnesia:add_table_copy fails with >> > {aborted,{already_exists,table_name,LastAddedNode}} >> > >> > Since the transaction lock ensures that no other process can add >> > another table copy, I guess this is a race condition where new table >> > copy node is not propagated to the schema on all nodes before the >> > function mnesia:add_table_copy returns. >> > >> > Karol >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From barcojie@REDACTED Fri Jul 8 11:39:27 2011 From: barcojie@REDACTED (Barco You) Date: Fri, 8 Jul 2011 17:39:27 +0800 Subject: [erlang-questions] [Erlang-Question] What do the items mean in the head line of Erlang shell prompt Message-ID: Every time I execute erl shell, there is a head line as following but I don't know the meanings. Could you please explain it to me. Thank you! Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Fri Jul 8 11:42:36 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Fri, 08 Jul 2011 11:42:36 +0200 Subject: [erlang-questions] [Erlang-Question] What do the items mean in the head line of Erlang shell prompt In-Reply-To: References: Message-ID: <4E16D10C.7010107@ubiquity.it> On 08/07/2011 11:39, Barco You wrote: > Every time I execute erl shell, there is a head line as following but > I don't know the meanings. Could you please explain it to me. Thank you! > > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) > This should help, it has quite some details: http://stackoverflow.com/questions/1182025/what-do-the-erlang-emulator-info-statements-mean Ciao, Carlo From kenneth.lundin@REDACTED Fri Jul 8 11:45:01 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 8 Jul 2011 11:45:01 +0200 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: References: Message-ID: I have sent them to Erlang Solutions so they can put them on the Erlang factory web-site. I assume that this will happen very soon and that we will be notified by Jodie. /Kenneth On Sun, Jun 26, 2011 at 8:20 PM, Paul Fisher wrote: > Kenneth, > > Can you publish your slides from your Erlang Factory talk on the OTP Roadmap? ?I apologize if you have already done so, I was looking on the Erlang Factory site and a link was not available. > > > -- > paul > > director, platform services > alertlogic, inc. > 713-484-8383 x2314 > > From jodie.burch@REDACTED Fri Jul 8 11:48:14 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Fri, 08 Jul 2011 10:48:14 +0100 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: Message-ID: Hi The slides are now on the website. Thanks Jodie On 08/07/2011 10:45, "Kenneth Lundin" wrote: > I have sent them to Erlang Solutions so they can put them on the > Erlang factory web-site. > I assume that this will happen very soon and that we will be notified by > Jodie. > > /Kenneth > > On Sun, Jun 26, 2011 at 8:20 PM, Paul Fisher wrote: >> Kenneth, >> >> Can you publish your slides from your Erlang Factory talk on the OTP Roadmap? >> ?I apologize if you have already done so, I was looking on the Erlang Factory >> site and a link was not available. >> >> >> -- >> paul >> >> director, platform services >> alertlogic, inc. >> 713-484-8383 x2314 >> >> From erlang@REDACTED Fri Jul 8 11:57:51 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Jul 2011 11:57:51 +0200 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: 2011/7/7 Fr?d?ric Trottier-H?bert : > The digest authentication you see at this point is something that's part of the HTTP protocol. It is technically unrelated to your application (although your application can see it through HTTP values) and the client-side of it is implemented by the browser. > > I would consider it highly impractical for how little control it gives you (even in terms of visual design) and in general security. The browser being in charge of it means the session handling is annoying (no way to disconnect, the browser just resubmits the data all the time for you, proxies become tedious, etc.) > > If you're okay for a longer read, here's what I know, in condensed format. Others can feel free to correct me. > > ====== > A QUICK GUIDE TO SAFER WEB AUTHENTICATION > ====== > Modern authentication schemes usually work a bit that way > ---------- > > i. The user registers to the website on an HTTPS page (so that the password being sent isn't observable by third parties) > > ii. The password is stored in a database after being hashed by bcrypt or scrypt with a convenient, but slow work factor as to make brute-force (if the DB is compromised) absolutely impractical. These algorithms store their own difficulty factor, so that if you want to up the complexity, it can be done quite simply even if you have many versions in the DB. > > iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, some of the SHA functions too. MD5 and SHA hashing functions were made to be really fast and we want to avoid that. The reason is that it makes it easier to brute-force passwords if the table is compromised. Protect your users first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and give each of them a work factor. If you take 100 millisecond to check a password (something that happens once per session, so it's fine to be slow) compared to 10 microseconds, it becomes a real pain for crackers to do their thing. During that time, you can warn your users to change their passwords in other services. > > iv. bcrypt and scrypt handle salting for you, as mentioned before. If you decided not to use them, do remember to salt your passwords to avoid easier rainbow table attacks. Do NOT reapply a hash function many times over a previous hash results. Because all hash functions have collisions, this means that as you go, you increase the chances of collisions. Re-salt at each iteration. Again, if you use bcrypt or scrypt, this is handled for you and you do not have to worry. Please use bcrypt or scrypt. > > v. If that wasn't obvious, use the hashing function of bcrypt, not the encryption that can be decrypted. You do NOT want to use encryption, but hashing. The reason is simple. If your system is compromised, you have to assume that your keys are also compromised. Encryption is unsafe for passwords. Hashing is the king. > > That being said, the actual steps of authentication then become: > > 1. The login page is an HTTPS page so that the password that will be sent isn't observable by third parties > 2. The user enters his own password, submits it > 3. The server hashes it through the same hashing function used when first saving the password > 4. Compare the two hash values. Note: you want this to be done in O(n) time so that it's impossible to know whether you get partial matches or not. This makes more sense for clear text passwords, but what the hell. > 4.a if they're different, the authentication failed > 4.b if they're the same, go to 5. > 5. Create some kind of session value in another table or wherever you want. The session will have a key that you can choose randomly (UUIDv4 + userid to avoid duplicates although the chances are minimal, as an example). > 6. Put the value in a cookie. You want that cookie to be set with the option HttpOnly to avoid having client-side javascript able to read it. Also give a reasonable timeout value to the cookie so the user has to re-authenticate from time to time. This means both server-time timeouts and client-side timeouts. > > When a user loads a page, check the session cookie you defined, and match its value to whatever you've got in your session table. If it matches, the user has an active session. If it doesn't, have the user authenticate again. > > Note that this scheme (and pretty much any other) is insecure against 3rd parties listening to the communications between you and the server. To solve this, move everything to HTTPS, although this is harder to optimise for performance. > > --- > Further considerations: > > A. Put a limit on how many times someone can try to log in. You can have their account locked for a few minutes in between each time. This is to keep people from trying to brute force accounts from the outside. > > B. HTTP cookies alone are not sufficient to know whether someone is authenticated or not. A specific kind of attack, named Cross-Site Request Forgery (CSRF) works by having someone doing an HTTP request for you, from a different domain. In these cases, the browser will automatically send the cookies along with the request (it can also be a javascript form that's auto-submitted). I've used this in the past to have a site administrator (a co-worker) close his own account without him noticing it. Woops. > > This means we need an external value, not coming from HTTP headers to make sure the request is good. > > In simple cases, you can try to check the referrer and make sure it's coming from your domain. However, flash components or HTTPS connections do not always have these settings, and if you have any of these pages on your site (and you will, if you do HTTPS authentication), you'll have to look somewhere else. > > The safe way to do requires a few elements: > > 1. Never do changes with GET requests. GET is idempotent and free of side effects > > 2. Do your changes with POST. This will require a user to use either a) Javascript, b) curl, wget or some other tool or c) an HTML form to send your data. This restricts the window of opportunity of malevolent users, who can no longer just post a link or an image to do CSRF. > > 3. This is the most important point, use something called a token. This token is a special value that is added to whatever form you generate. When the user sends a POST query (or anything requiring server-side changes) to you, check that the token is there and that it is valid. The best tokens will have a one-time use, but they're fairly impractical when many different queries can be done without refreshing a page. My favourite way to handle these otherwise is to create different tokens for different namespaces. The admin panel will have its own token, the chat system its own token, the user settings will also have its own, etc. > > The tokens have to time out at some point to make sure that someone who steals them can not damage the account for too long. Pick between a few seconds to a few hours based on how long you expect your user to stay on a single page. > > 4. Ask for the user password when changing vital information, such as the e-mail attached to the account, the password itself, buying an item, etc. > > C. Never roll out your own encryption schemes or hash functions. Let mathematicians specialised in cryptography do their thing. Thank you - that was very helpful. Actually I have implement RSA and a few other things in pure Erlang using bignums - one property of these is that they are slow - (which is desirable by point iii) above I'd never thought that slowness could be an advantage. /Joe > > ======== > > That should be about it for a quick guide. There are more complex issues to tackle for good authentication and security of user data, but as a basic intro, this should be okay. I hope this is helpful. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: > >> Slightly off topic. But I want to make an erlang web site. >> >> 1) How does web authentication work? >> >> Let's assume something like: >> >> ? http://en.wikipedia.org/wiki/Digest_access_authentication >> >> This is easy to understand. >> >> What I don't understand is what happens if the session socket is closed. >> Handshaking tales place over an open socket and the client is >> authenticated - this >> is easy to understand. >> >> What happens if the socket is closed, and reopened in a subsequent request? >> Does the server set and receive a session cookie? Does the client remember and >> replay the authentication protocol? >> >> How does this work? >> >> 2) I want to make a web thing that requires the user to authenticate themself. >> >> Should I: >> >> ? ?a) Roll my own (some MD5 + cookies should do the job) >> ? ?b) Implement ?http://en.wikipedia.org/wiki/Digest_access_authentication >> ? ?c) Something else? >> >> Seems like for a real web site there is a lot of cruft involved >> preventing spammers, >> false-accounts, forgotten-passwords etc. can I get all of this for >> free by getting >> authentication credentials via goole/facebook or something? Is this >> what OpenID does? >> >> Finally is this entire authentication-user management-forgot my >> password built-in >> to any of the popular erlang web servers? >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From pfisher@REDACTED Fri Jul 8 12:46:54 2011 From: pfisher@REDACTED (Paul Fisher) Date: Fri, 8 Jul 2011 05:46:54 -0500 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: Message-ID: Thank you both! -- paul director, platform services alertlogic, inc. pfisher@REDACTED 713-484-8383 x2314 On 7/8/11 4:48 AM, "Jodie Burch" wrote: >Hi > >The slides are now on the website. > >Thanks >Jodie > > >On 08/07/2011 10:45, "Kenneth Lundin" wrote: > >> I have sent them to Erlang Solutions so they can put them on the >> Erlang factory web-site. >> I assume that this will happen very soon and that we will be notified by >> Jodie. >> >> /Kenneth >> >> On Sun, Jun 26, 2011 at 8:20 PM, Paul Fisher >>wrote: >>> Kenneth, >>> >>> Can you publish your slides from your Erlang Factory talk on the OTP >>>Roadmap? >>> I apologize if you have already done so, I was looking on the Erlang >>>Factory >>> site and a link was not available. >>> >>> >>> -- >>> paul >>> >>> director, platform services >>> alertlogic, inc. >>> 713-484-8383 x2314 >>> >>> > > From fred.hebert@REDACTED Fri Jul 8 12:54:48 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 8 Jul 2011 06:54:48 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: <08DE64DD-8946-4A27-9562-18CA415FEC30@erlang-solutions.com> Slowness is only helpful when it is slow across all languages, though. If your RSA is slow in Erlang, but very fast in C, Fortran or whatever language, then it doesn't matter. Someone who finds your database (lost backup, infiltrating your servers, SQL injection, etc) will use the language they want to run these algorithms. Again, that's why bcrypt and scrypt are so useful as hash functions. The work factor you give them makes it sure that *any* implementation is going to take a lot of time to do its thing. But yeah, that algorithmic slowness is warranted is a funny thing. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-08, at 05:57 AM, Joe Armstrong wrote: > 2011/7/7 Fr?d?ric Trottier-H?bert : >> The digest authentication you see at this point is something that's part of the HTTP protocol. It is technically unrelated to your application (although your application can see it through HTTP values) and the client-side of it is implemented by the browser. >> >> I would consider it highly impractical for how little control it gives you (even in terms of visual design) and in general security. The browser being in charge of it means the session handling is annoying (no way to disconnect, the browser just resubmits the data all the time for you, proxies become tedious, etc.) >> >> If you're okay for a longer read, here's what I know, in condensed format. Others can feel free to correct me. >> >> ====== >> A QUICK GUIDE TO SAFER WEB AUTHENTICATION >> ====== >> Modern authentication schemes usually work a bit that way >> ---------- >> >> i. The user registers to the website on an HTTPS page (so that the password being sent isn't observable by third parties) >> >> ii. The password is stored in a database after being hashed by bcrypt or scrypt with a convenient, but slow work factor as to make brute-force (if the DB is compromised) absolutely impractical. These algorithms store their own difficulty factor, so that if you want to up the complexity, it can be done quite simply even if you have many versions in the DB. >> >> iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, some of the SHA functions too. MD5 and SHA hashing functions were made to be really fast and we want to avoid that. The reason is that it makes it easier to brute-force passwords if the table is compromised. Protect your users first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and give each of them a work factor. If you take 100 millisecond to check a password (something that happens once per session, so it's fine to be slow) compared to 10 microseconds, it becomes a real pain for crackers to do their thing. During that time, you can warn your users to change their passwords in other services. >> >> iv. bcrypt and scrypt handle salting for you, as mentioned before. If you decided not to use them, do remember to salt your passwords to avoid easier rainbow table attacks. Do NOT reapply a hash function many times over a previous hash results. Because all hash functions have collisions, this means that as you go, you increase the chances of collisions. Re-salt at each iteration. Again, if you use bcrypt or scrypt, this is handled for you and you do not have to worry. Please use bcrypt or scrypt. >> >> v. If that wasn't obvious, use the hashing function of bcrypt, not the encryption that can be decrypted. You do NOT want to use encryption, but hashing. The reason is simple. If your system is compromised, you have to assume that your keys are also compromised. Encryption is unsafe for passwords. Hashing is the king. >> >> That being said, the actual steps of authentication then become: >> >> 1. The login page is an HTTPS page so that the password that will be sent isn't observable by third parties >> 2. The user enters his own password, submits it >> 3. The server hashes it through the same hashing function used when first saving the password >> 4. Compare the two hash values. Note: you want this to be done in O(n) time so that it's impossible to know whether you get partial matches or not. This makes more sense for clear text passwords, but what the hell. >> 4.a if they're different, the authentication failed >> 4.b if they're the same, go to 5. >> 5. Create some kind of session value in another table or wherever you want. The session will have a key that you can choose randomly (UUIDv4 + userid to avoid duplicates although the chances are minimal, as an example). >> 6. Put the value in a cookie. You want that cookie to be set with the option HttpOnly to avoid having client-side javascript able to read it. Also give a reasonable timeout value to the cookie so the user has to re-authenticate from time to time. This means both server-time timeouts and client-side timeouts. >> >> When a user loads a page, check the session cookie you defined, and match its value to whatever you've got in your session table. If it matches, the user has an active session. If it doesn't, have the user authenticate again. >> >> Note that this scheme (and pretty much any other) is insecure against 3rd parties listening to the communications between you and the server. To solve this, move everything to HTTPS, although this is harder to optimise for performance. >> >> --- >> Further considerations: >> >> A. Put a limit on how many times someone can try to log in. You can have their account locked for a few minutes in between each time. This is to keep people from trying to brute force accounts from the outside. >> >> B. HTTP cookies alone are not sufficient to know whether someone is authenticated or not. A specific kind of attack, named Cross-Site Request Forgery (CSRF) works by having someone doing an HTTP request for you, from a different domain. In these cases, the browser will automatically send the cookies along with the request (it can also be a javascript form that's auto-submitted). I've used this in the past to have a site administrator (a co-worker) close his own account without him noticing it. Woops. >> >> This means we need an external value, not coming from HTTP headers to make sure the request is good. >> >> In simple cases, you can try to check the referrer and make sure it's coming from your domain. However, flash components or HTTPS connections do not always have these settings, and if you have any of these pages on your site (and you will, if you do HTTPS authentication), you'll have to look somewhere else. >> >> The safe way to do requires a few elements: >> >> 1. Never do changes with GET requests. GET is idempotent and free of side effects >> >> 2. Do your changes with POST. This will require a user to use either a) Javascript, b) curl, wget or some other tool or c) an HTML form to send your data. This restricts the window of opportunity of malevolent users, who can no longer just post a link or an image to do CSRF. >> >> 3. This is the most important point, use something called a token. This token is a special value that is added to whatever form you generate. When the user sends a POST query (or anything requiring server-side changes) to you, check that the token is there and that it is valid. The best tokens will have a one-time use, but they're fairly impractical when many different queries can be done without refreshing a page. My favourite way to handle these otherwise is to create different tokens for different namespaces. The admin panel will have its own token, the chat system its own token, the user settings will also have its own, etc. >> >> The tokens have to time out at some point to make sure that someone who steals them can not damage the account for too long. Pick between a few seconds to a few hours based on how long you expect your user to stay on a single page. >> >> 4. Ask for the user password when changing vital information, such as the e-mail attached to the account, the password itself, buying an item, etc. >> >> C. Never roll out your own encryption schemes or hash functions. Let mathematicians specialised in cryptography do their thing. > > Thank you - that was very helpful. > > Actually I have implement RSA > and a few other things in pure Erlang using bignums - one property > of these is that they are slow - (which is desirable by point iii) above > I'd never thought that slowness could be an advantage. > > /Joe > >> >> ======== >> >> That should be about it for a quick guide. There are more complex issues to tackle for good authentication and security of user data, but as a basic intro, this should be okay. I hope this is helpful. >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> >> >> On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: >> >>> Slightly off topic. But I want to make an erlang web site. >>> >>> 1) How does web authentication work? >>> >>> Let's assume something like: >>> >>> http://en.wikipedia.org/wiki/Digest_access_authentication >>> >>> This is easy to understand. >>> >>> What I don't understand is what happens if the session socket is closed. >>> Handshaking tales place over an open socket and the client is >>> authenticated - this >>> is easy to understand. >>> >>> What happens if the socket is closed, and reopened in a subsequent request? >>> Does the server set and receive a session cookie? Does the client remember and >>> replay the authentication protocol? >>> >>> How does this work? >>> >>> 2) I want to make a web thing that requires the user to authenticate themself. >>> >>> Should I: >>> >>> a) Roll my own (some MD5 + cookies should do the job) >>> b) Implement http://en.wikipedia.org/wiki/Digest_access_authentication >>> c) Something else? >>> >>> Seems like for a real web site there is a lot of cruft involved >>> preventing spammers, >>> false-accounts, forgotten-passwords etc. can I get all of this for >>> free by getting >>> authentication credentials via goole/facebook or something? Is this >>> what OpenID does? >>> >>> Finally is this entire authentication-user management-forgot my >>> password built-in >>> to any of the popular erlang web servers? >>> >>> Cheers >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> From magnus.falk@REDACTED Fri Jul 8 16:28:55 2011 From: magnus.falk@REDACTED (Magnus Falk) Date: Fri, 8 Jul 2011 16:28:55 +0200 Subject: [erlang-questions] Examining Erlang crash dumps - how to account for all memory? Message-ID: This is a copy of a question I asked over at StackOverflow that noone has been able to answer yet: http://stackoverflow.com/questions/6616101/examining-erlang-crash-dumps-how-to-account-for-all-memory I've been poring over this Erlang crash dump where the VM has run out of heap memory. The problem is that there is no obvious culprit allocating all that memory. Using some serious black awk magic I've summed up the fields Stack+heap, OldHeap, Heap unused and OldHeap unused for each process and ranked them by memory usage. The problem is that this number doesn't come even close to the number that is representing the total memory for all the processes processes_used according to the Erlang crash dump guide. I've already tried the Crashdump Viewer and either I'm missing something or there isn't much help there for my kind of problem. The number I get is 525 MB whereas the processes_used value is at 1348 MB. Where can I find the rest of the memory? Cheers, Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From montuori@REDACTED Fri Jul 8 16:36:14 2011 From: montuori@REDACTED (kevin montuori) Date: Fri, 8 Jul 2011 10:36:14 -0400 Subject: [erlang-questions] Addition to slave.erl Message-ID: Hi all -- I have a use for slave:start/3 but need to pass additional flags to the rsh (actually ssh) command. To this end, I created a new slave:start/4 (and rejigged the existing start/4,5 functions) to allow for start(Host, Name, Args, RshArgs). For consistency I added a slave:start_link/4 as well. I have no idea if this is useful to others. I've attached the diffs for slave.erl and slave.xml; I wasn't sure how to augment the unit tests appropriately but would certainly welcome advice. Disclaimer: I'm relatively new to Erlang and am open to comments, style advice, &c.. Cheers! k. -- kevin montuori -------------- next part -------------- A non-text attachment was scrubbed... Name: slave.erl.diff Type: application/octet-stream Size: 3561 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: slave.xml.diff Type: application/octet-stream Size: 2591 bytes Desc: not available URL: From ulf.wiger@REDACTED Fri Jul 8 16:44:37 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 8 Jul 2011 16:44:37 +0200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> Message-ID: The great thing about Open Source is that people *may* help you out if you are able to pose a challenge that they find interesting enough. The key is *may*, though. If people don't bite, there is no point in chiding them for not helping you (for free). As it now stands, several very knowledgeable people have taken their precious time to challenge the premises of the experiment. This is also extremely valuable help, which would carry a pretty hefty price tag, if you should seek it through traditional consulting services (probably from the same type of people). If you decide to go ahead with your experiment (with or without community assistance), you are now better prepared for the kind of answers people will be looking for. At this point in my career, I care little for micro-benchmarks, unless they have been exactly tailored to answer the specific questions I have for my next project. Every sizeable project should start with identifying the critical challenges, based on your problem, non-functional requirements, choice of main technology, architecture, hardware, environmental restrictions, etc. Even then, one must be ruthlessly honest when evaluating the results (as well as the challenges themselves!), since it is so easy to stack the experiments, and apply wishful thinking to the results, so that you can forge ahead with the solutions you wanted to try out in the first place. Having said this, I invite anyone who goes through that kind of exercise to share their results. Not only will it help you evaluate the experiment honestly; it will increase the store of experiments that can be copied and tailored to the specific challenges of the next project. BR, Ulf W On 6 Jul 2011, at 22:55, Amy Lear wrote: > On Wed, Jul 6, 2011 at 12:47 PM, Zabrane Mickael wrote: >> >> Le 6 juil. 2011 ? 21:38, Jon Watte a ?crit : >> >>> Another ambiguous part of the problem as stated: >>> The way the question is worded, I could write a TCP server that accepts a connection, then spits out a static blob of text consisting of static HTTP headers and the contents of the file -- I don't even need to parse the request, because this file is the only thing that the server will serve. >> >> Funny analysis, but that's a toy web server. In the challenge list, I asked for a real web server instead (I've cited Misultin, Yaws ...). > > A toy web server for a toy benchmark. Seems like a perfect match. > > Again: You need to define your objectives here, or we're not going to > get anywhere. If your objective is simply "I want to test webservers > according to arbitrary criteria that are not up for discussion", then > I think we're done here. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Fri Jul 8 17:06:32 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 8 Jul 2011 17:06:32 +0200 Subject: [erlang-questions] Asynchronous I/O - Getting documentation. In-Reply-To: References: Message-ID: Well, IO is really synchronous from the point of view of the caller. That is, the erlang process requesting input or output is blocked while waiting for the service to be performed. In erlang terms, it performs a synchronous call (i.e. sends a message and waits for the result in a "selective receive" statement) to another erlang process. The request may be dispatched in different ways depending on the system. If the erlang node in question doesn't have a terminal, the request may be served transparently on another node, which does. For example, if a function is executed on a remote erlang node via an rpc:call(), any io effected by the called function will be automatically routed back to the calling node. As a result, if you call rpc:call(OtherNode, io, format, ["hello~n"]), the output, "hello", will appear in the shell of your own node - not on OtherNode. This is very useful, not least when debugging embedded systems. The runtime system itself either performs non-blocking IO or dispatches an OS thread to perform blocking IO. This is an implementation detail that the erlang program itself has no control over. BR, Ulf W On 7 Jul 2011, at 04:37, Ale wrote: > Hello All, > > I'm looking for documentation to expand my knowledge of the > implementation of how asynchronous I/O works; from wikipedia. > > """This approach is also used in the Erlang programming language > runtime system. The Erlang virtual machine uses asynchronous IO using > a small pool of only a few threads or sometimes just one process, to > handle IO from up to millions of Erlang processes. IO handling in each > process is written mostly using blocking synchronous I/O. This way > high performance of asynchronous I/O is merged with simplicity of > normal IO. Many IO problems in Erlang are mapped to message passing, > which can be easily processed using built-in selective receive.""" > > I'm interested in learning how the scheduler works, and how the > process which handle IO work. > > Thanks, > -- > Ale. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jack@REDACTED Fri Jul 8 17:45:16 2011 From: jack@REDACTED (Jack Moffitt) Date: Fri, 8 Jul 2011 09:45:16 -0600 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: > Actually I have implement RSA > and a few other things in pure Erlang using bignums - one property > of these is that they are slow - (which is desirable by point iii) above > I'd never thought that slowness could be an advantage. If you are interested in slowness being a feature, you should read the Bitcoin paper and the other related work on "forced work". It turns out to be a quite useful primitive for certain things. In Bitcoin for example, the forced work is also scalable, so the minting of new coins in the system happens at an approximately constant rate despite technology upgrades and the influx of new workers. jack. From jack@REDACTED Fri Jul 8 17:48:53 2011 From: jack@REDACTED (Jack Moffitt) Date: Fri, 8 Jul 2011 09:48:53 -0600 Subject: [erlang-questions] Examining Erlang crash dumps - how to account for all memory? In-Reply-To: References: Message-ID: > I've already tried the Crashdump Viewer and either I'm missing something or > there isn't much help there for my kind of problem. > > The number I get is 525 MB whereas the processes_used value is at 1348 MB. > Where can I find the rest of the memory? Usually when my VMs run out of memory it turns out to be a really large process mailbox. Try looking at the message queue lengths instead of just the heap. Also note that Erlang's GC is generational, so when it runs out, it must allocate a new block larger than the old block. So if it's got a block at 500MB and tries to allocate 600MB for the new generation, well, that's getting close to the number you posted. jack. From erlang@REDACTED Fri Jul 8 17:55:43 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Jul 2011 17:55:43 +0200 Subject: [erlang-questions] web authentication In-Reply-To: <08DE64DD-8946-4A27-9562-18CA415FEC30@erlang-solutions.com> References: <08DE64DD-8946-4A27-9562-18CA415FEC30@erlang-solutions.com> Message-ID: 2011/7/8 Fr?d?ric Trottier-H?bert : > Slowness is only helpful when it is slow across all languages, though. If your RSA is slow in Erlang, but very fast in C, Fortran or whatever language, then it doesn't matter. Someone who finds your database (lost backup, infiltrating your servers, SQL injection, etc) will use the language they want to run these algorithms. Again, that's why bcrypt and scrypt are so useful as hash functions. The work factor you give them makes it sure that *any* implementation is going to take a lot of time to do its thing. Point taken - though there is no C code (yet) to decode my RSA certificates (which are just term_to_binary({A,B,C}) where A B and C are bignums for computing A^B mod(C) :-) > But yeah, that algorithmic slowness is warranted is a funny thing. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-08, at 05:57 AM, Joe Armstrong wrote: > >> 2011/7/7 Fr?d?ric Trottier-H?bert : >>> The digest authentication you see at this point is something that's part of the HTTP protocol. It is technically unrelated to your application (although your application can see it through HTTP values) and the client-side of it is implemented by the browser. >>> >>> I would consider it highly impractical for how little control it gives you (even in terms of visual design) and in general security. The browser being in charge of it means the session handling is annoying (no way to disconnect, the browser just resubmits the data all the time for you, proxies become tedious, etc.) >>> >>> If you're okay for a longer read, here's what I know, in condensed format. Others can feel free to correct me. >>> >>> ====== >>> A QUICK GUIDE TO SAFER WEB AUTHENTICATION >>> ====== >>> Modern authentication schemes usually work a bit that way >>> ---------- >>> >>> i. The user registers to the website on an HTTPS page (so that the password being sent isn't observable by third parties) >>> >>> ii. The password is stored in a database after being hashed by bcrypt or scrypt with a convenient, but slow work factor as to make brute-force (if the DB is compromised) absolutely impractical. These algorithms store their own difficulty factor, so that if you want to up the complexity, it can be done quite simply even if you have many versions in the DB. >>> >>> iii. Avoid using MD5 or SHA-x hashing functions. MD5 is collision-prone, some of the SHA functions too. MD5 and SHA hashing functions were made to be really fast and we want to avoid that. The reason is that it makes it easier to brute-force passwords if the table is compromised. Protect your users first. Bcrypt and Scrypt, by comparison, will salt the passwords for you and give each of them a work factor. If you take 100 millisecond to check a password (something that happens once per session, so it's fine to be slow) compared to 10 microseconds, it becomes a real pain for crackers to do their thing. During that time, you can warn your users to change their passwords in other services. >>> >>> iv. bcrypt and scrypt handle salting for you, as mentioned before. If you decided not to use them, do remember to salt your passwords to avoid easier rainbow table attacks. Do NOT reapply a hash function many times over a previous hash results. Because all hash functions have collisions, this means that as you go, you increase the chances of collisions. Re-salt at each iteration. Again, if you use bcrypt or scrypt, this is handled for you and you do not have to worry. Please use bcrypt or scrypt. >>> >>> v. If that wasn't obvious, use the hashing function of bcrypt, not the encryption that can be decrypted. You do NOT want to use encryption, but hashing. The reason is simple. If your system is compromised, you have to assume that your keys are also compromised. Encryption is unsafe for passwords. Hashing is the king. >>> >>> That being said, the actual steps of authentication then become: >>> >>> 1. The login page is an HTTPS page so that the password that will be sent isn't observable by third parties >>> 2. The user enters his own password, submits it >>> 3. The server hashes it through the same hashing function used when first saving the password >>> 4. Compare the two hash values. Note: you want this to be done in O(n) time so that it's impossible to know whether you get partial matches or not. This makes more sense for clear text passwords, but what the hell. >>> 4.a if they're different, the authentication failed >>> 4.b if they're the same, go to 5. >>> 5. Create some kind of session value in another table or wherever you want. The session will have a key that you can choose randomly (UUIDv4 + userid to avoid duplicates although the chances are minimal, as an example). >>> 6. Put the value in a cookie. You want that cookie to be set with the option HttpOnly to avoid having client-side javascript able to read it. Also give a reasonable timeout value to the cookie so the user has to re-authenticate from time to time. This means both server-time timeouts and client-side timeouts. >>> >>> When a user loads a page, check the session cookie you defined, and match its value to whatever you've got in your session table. If it matches, the user has an active session. If it doesn't, have the user authenticate again. >>> >>> Note that this scheme (and pretty much any other) is insecure against 3rd parties listening to the communications between you and the server. To solve this, move everything to HTTPS, although this is harder to optimise for performance. >>> >>> --- >>> Further considerations: >>> >>> A. Put a limit on how many times someone can try to log in. You can have their account locked for a few minutes in between each time. This is to keep people from trying to brute force accounts from the outside. >>> >>> B. HTTP cookies alone are not sufficient to know whether someone is authenticated or not. A specific kind of attack, named Cross-Site Request Forgery (CSRF) works by having someone doing an HTTP request for you, from a different domain. In these cases, the browser will automatically send the cookies along with the request (it can also be a javascript form that's auto-submitted). I've used this in the past to have a site administrator (a co-worker) close his own account without him noticing it. Woops. >>> >>> This means we need an external value, not coming from HTTP headers to make sure the request is good. >>> >>> In simple cases, you can try to check the referrer and make sure it's coming from your domain. However, flash components or HTTPS connections do not always have these settings, and if you have any of these pages on your site (and you will, if you do HTTPS authentication), you'll have to look somewhere else. >>> >>> The safe way to do requires a few elements: >>> >>> 1. Never do changes with GET requests. GET is idempotent and free of side effects >>> >>> 2. Do your changes with POST. This will require a user to use either a) Javascript, b) curl, wget or some other tool or c) an HTML form to send your data. This restricts the window of opportunity of malevolent users, who can no longer just post a link or an image to do CSRF. >>> >>> 3. This is the most important point, use something called a token. This token is a special value that is added to whatever form you generate. When the user sends a POST query (or anything requiring server-side changes) to you, check that the token is there and that it is valid. The best tokens will have a one-time use, but they're fairly impractical when many different queries can be done without refreshing a page. My favourite way to handle these otherwise is to create different tokens for different namespaces. The admin panel will have its own token, the chat system its own token, the user settings will also have its own, etc. >>> >>> The tokens have to time out at some point to make sure that someone who steals them can not damage the account for too long. Pick between a few seconds to a few hours based on how long you expect your user to stay on a single page. >>> >>> 4. Ask for the user password when changing vital information, such as the e-mail attached to the account, the password itself, buying an item, etc. >>> >>> C. Never roll out your own encryption schemes or hash functions. Let mathematicians specialised in cryptography do their thing. >> >> Thank you - that was very helpful. >> >> Actually I have implement RSA >> and a few other things in pure Erlang using bignums - one property >> of these is that they are slow - (which is desirable by point iii) above >> I'd never thought that slowness could be an advantage. >> >> /Joe >> >>> >>> ======== >>> >>> That should be about it for a quick guide. There are more complex issues to tackle for good authentication and security of user data, but as a basic intro, this should be okay. I hope this is helpful. >>> >>> -- >>> Fred H?bert >>> http://www.erlang-solutions.com >>> >>> >>> >>> On 2011-07-07, at 15:29 PM, Joe Armstrong wrote: >>> >>>> Slightly off topic. But I want to make an erlang web site. >>>> >>>> 1) How does web authentication work? >>>> >>>> Let's assume something like: >>>> >>>> ? http://en.wikipedia.org/wiki/Digest_access_authentication >>>> >>>> This is easy to understand. >>>> >>>> What I don't understand is what happens if the session socket is closed. >>>> Handshaking tales place over an open socket and the client is >>>> authenticated - this >>>> is easy to understand. >>>> >>>> What happens if the socket is closed, and reopened in a subsequent request? >>>> Does the server set and receive a session cookie? Does the client remember and >>>> replay the authentication protocol? >>>> >>>> How does this work? >>>> >>>> 2) I want to make a web thing that requires the user to authenticate themself. >>>> >>>> Should I: >>>> >>>> ? ?a) Roll my own (some MD5 + cookies should do the job) >>>> ? ?b) Implement ?http://en.wikipedia.org/wiki/Digest_access_authentication >>>> ? ?c) Something else? >>>> >>>> Seems like for a real web site there is a lot of cruft involved >>>> preventing spammers, >>>> false-accounts, forgotten-passwords etc. can I get all of this for >>>> free by getting >>>> authentication credentials via goole/facebook or something? Is this >>>> what OpenID does? >>>> >>>> Finally is this entire authentication-user management-forgot my >>>> password built-in >>>> to any of the popular erlang web servers? >>>> >>>> Cheers >>>> >>>> /Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> > > From erlang@REDACTED Fri Jul 8 17:58:50 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Jul 2011 17:58:50 +0200 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: 2011/7/8 Jack Moffitt : >> Actually I have implement RSA >> and a few other things in pure Erlang using bignums - one property >> of these is that they are slow - (which is desirable by point iii) above >> I'd never thought that slowness could be an advantage. > > If you are interested in slowness being a feature, you should read the > Bitcoin paper and the other related work on "forced work". It turns > out to be a quite useful primitive for certain things. In Bitcoin for > example, the forced work is also scalable, so the minting of new coins > in the system happens at an approximately constant rate despite > technology upgrades and the influx of new workers. > > jack. Ummm - now wait a moment I think I can prove that NP = P - Yes - Eureka Unfortunately the margin isn't big enough ... /Joe From anthonym@REDACTED Fri Jul 8 19:25:07 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Fri, 8 Jul 2011 10:25:07 -0700 Subject: [erlang-questions] OTP roadmap slides from Erlang Factory? In-Reply-To: References: Message-ID: <20110708172507.GC26933@alumni.caltech.edu> Will more videos be available over time? I see a few but not many. Also, what was the results of the EEP panel? Are there process changes or an updated list of EEPs with priorities? Also, does the list on http://www.erlang.org/eeps/ get updated frequently? I understand from a comment on EQ yesterday that EEP36 will be in R15, but don't see R15 listed anywhere and also see no way to determine what might be being worked on? Anyway, sorry if this is off topic, but I just noticed that the panel happened and was curious what went on there. -Anthony On Fri, Jul 08, 2011 at 10:48:14AM +0100, Jodie Burch wrote: > Hi > > The slides are now on the website. > > Thanks > Jodie > > > On 08/07/2011 10:45, "Kenneth Lundin" wrote: > > > I have sent them to Erlang Solutions so they can put them on the > > Erlang factory web-site. > > I assume that this will happen very soon and that we will be notified by > > Jodie. > > > > /Kenneth > > > > On Sun, Jun 26, 2011 at 8:20 PM, Paul Fisher wrote: > >> Kenneth, > >> > >> Can you publish your slides from your Erlang Factory talk on the OTP Roadmap? > >> ?I apologize if you have already done so, I was looking on the Erlang Factory > >> site and a link was not available. > >> > >> > >> -- > >> paul > >> > >> director, platform services > >> alertlogic, inc. > >> 713-484-8383 x2314 > >> > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From jwatte@REDACTED Sat Jul 9 06:59:37 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 8 Jul 2011 21:59:37 -0700 Subject: [erlang-questions] Mysterious boot failure In-Reply-To: References: Message-ID: Thanks! The directory is the same (set by the start script) but I'll have to check the PAM session limits. That's a good lead! Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jul 6, 2011 at 1:29 PM, Juan Jose Comellas wrote: > I don't know if this is the cause, but if you're setting specific values > for the system limits in /etc/security/limits.conf for your user, make sure > that they're being set by PAM when executing via sudo. You can check this in > /etc/pam.d/sudo. You should look for a line like the following one and make > sure that it's not commented out: > > session required pam_limits.so > > Also, I'd also check that you're starting the Erlang VM in the correct > directory (i.e. one where your user has permissions to read). > > Juanjo > > > On Wed, Jul 6, 2011 at 5:05 PM, Jon Watte wrote: > >> We're running a cluster of 11 Erlang nodes, running on R13B3 using Ubuntu >> 10.04 LTS 64-bit. >> The application is installed in a subdirectory of the home directory of a >> particular user. We start the application using nohup, and with stdio/stderr >> re-direct to a log file. >> We're seeing a mysterious crash in some cases (see below) >> >> If we're logged in as the appropriate user, everything starts fine. >> If we're logged in as root, and do "sudo -u username " then >> beam.smp crashes with "init terminating in do_boot". >> Why is this? I can't figure out why it's crashing like this. >> >> {no error logger present) error: "Error in process <0.2.0> with exit >> value: >> {badarg,[{erl_prim_loader,check_file_result,3},{erl_prim_loader,check_file_result,3},{init,get_boot,1},{init,get_boot,2},{init,do_boot,3}]}\n" >> init terminating in do_boot () >> >> >> Googling on this error shows only two other web resources; one an IRC chat >> with basho, not getting an answer, and one a message on this mailing list, >> also not getting an answer. >> Reading the code, it seems as if the only way this will happen is if there >> is an error, but the "Func" argument or "Reason" argument to >> check_file_result is somehow not an atom. I don't, however, see how that >> could be happening. >> The other question is also why the boot script would not be >> visible/available at that point. The particular user has no .bashrc or other >> such init script. >> >> >> The actual start script (with secrets sanitized) is: >> >> nohup erl +K true -noshell \ >> -env ERL_MAX_PORTS 200500 \ >> +W w +P 1001001 \ >> -boot start_sasl \ >> -cluster leader@REDACTED \ >> -kernel inet_dist_listen_min 50000 inet_dist_listen_max 50009 \ >> -name "$node@$hostname" -setcookie "some secret" \ >> -callout svc_url '"http://service/%%.php"' \ >> -s launcher -extra $role \ >> "$logdir/$node-logfile.txt" 2>&1 & >> >> >> >> Here's the crash-dump analysis (not very useful): >> >> Slogan init terminating in do_boot () >> Node name 'mqnode@REDACTED' >> Crashdump created on Wed Jul 6 10:23:58 2011 >> System version Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] >> [rq:8] [async-threads:0] [hipe] [kernel-poll:true] >> Compiled Fri Apr 9 12:29:55 2010 >> Memory allocated 141146456 bytes >> Atoms 5690 >> Processes 35 >> ETS tables 17 >> Funs 357 >> >> -- >> Americans might object: there is no way we would sacrifice our living >> standards for the benefit of people in the rest of the world. Nevertheless, >> whether we get there willingly or not, we shall soon have lower consumption >> rates, because our present rates are unsustainable. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Jul 9 07:14:40 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 8 Jul 2011 22:14:40 -0700 Subject: [erlang-questions] Examining Erlang crash dumps - how to account for all memory? In-Reply-To: References: Message-ID: Certain objects, such as binaries and ets tables, live in different memory than that of each process. If you have lots of those, then that might explain the rest of the memory. Also, if you look at what each process is doing, is any process "garbing"? If so, it probably is using more memory temporarily for the new block it's compacting into. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Jul 8, 2011 at 7:28 AM, Magnus Falk wrote: > This is a copy of a question I asked over at StackOverflow that noone has > been able to answer yet: > http://stackoverflow.com/questions/6616101/examining-erlang-crash-dumps-how-to-account-for-all-memory > > I've been poring over this Erlang crash dump where the VM has run out of > heap memory. The problem is that there is no obvious culprit allocating all > that memory. > > Using some serious black awk magic I've summed up the fields Stack+heap, > OldHeap, Heap unused and OldHeap unused for each process and ranked them by > memory usage. The problem is that this number doesn't come even close to the > number that is representing the total memory for all the processes > processes_used according to the Erlang crash dump guide. > > I've already tried the Crashdump Viewer and either I'm missing something or > there isn't much help there for my kind of problem. > > The number I get is 525 MB whereas the processes_used value is at 1348 MB. > Where can I find the rest of the memory? > > Cheers, > Magnus > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Jul 9 07:20:42 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 8 Jul 2011 22:20:42 -0700 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: You've already gotten some good answers; however, having worked extensively with this myself, I recommend: 1) Use HTTPS for all traffic. Computers are fast. Crypto is cheap. Cookie theft is a reality on open channels. 2) Use Basic-auth over HTTP -- this sends name and password, base-64-encoded. 3) HTTP is stateless. The auth will be sent with each and every request. The fact that HTTP/1.1 allows you to keep the physical TCP connection open and send another request is purely an optimization; you cannot assume anything about the next request that comes in. In fact, a HTTP gateway may multiplex multiple internal users to a single external connection, and send different credentials (including no credentials) with each request. The stateless nature of HTTP is what makes it scale really, really well for certain problem domains and usage patterns. It's also what makes it a pain in the butt when certain kinds of efficiencies are important. This area is fraught with more distress than you would normally think. I believe this is because old-school networking people sometimes haven't yet realized the benefits of HTTP as currently implemented and optimized; meanwhile, web-only people often don't understand that there are cases where HTTP is NOT the right solution. Sitting in the middle is fun and frustrating at the same time ;-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Jul 7, 2011 at 12:29 PM, Joe Armstrong wrote: > Slightly off topic. But I want to make an erlang web site. > > 1) How does web authentication work? > > Let's assume something like: > > http://en.wikipedia.org/wiki/Digest_access_authentication > > This is easy to understand. > > What I don't understand is what happens if the session socket is closed. > Handshaking tales place over an open socket and the client is > authenticated - this > is easy to understand. > > What happens if the socket is closed, and reopened in a subsequent request? > Does the server set and receive a session cookie? Does the client remember > and > replay the authentication protocol? > > How does this work? > > 2) I want to make a web thing that requires the user to authenticate > themself. > > Should I: > > a) Roll my own (some MD5 + cookies should do the job) > b) Implement http://en.wikipedia.org/wiki/Digest_access_authentication > c) Something else? > > Seems like for a real web site there is a lot of cruft involved > preventing spammers, > false-accounts, forgotten-passwords etc. can I get all of this for > free by getting > authentication credentials via goole/facebook or something? Is this > what OpenID does? > > Finally is this entire authentication-user management-forgot my > password built-in > to any of the popular erlang web servers? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Jul 9 07:40:23 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 9 Jul 2011 09:40:23 +0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: On Sat, Jul 9, 2011 at 9:20 AM, Jon Watte wrote: > You've already gotten some good answers; however, having worked extensively > with this myself, I recommend: > > 1) Use HTTPS for all traffic. Computers are fast. Crypto is cheap. Cookie > theft is a reality on open channels. > This is not as clear as it seems to. 1) HTTPS add delay. It is not a blocking problem for social network/wiki, but it is a profit penalty for online shop 2) HTTPS still has issues with Internet Explorer, when some code is HTTP and some is HTTPS. And when something is HTTPS, but IE thinks it is HTTP > 2) Use Basic-auth over HTTP -- this sends name and password, > base-64-encoded. > Sorry, but it is very, very bad recommendation. You are afraid of cookie stealing and this is why you recommend to send name and password plaintext on each request. What for? From isaacbfsanders@REDACTED Sat Jul 9 15:18:59 2011 From: isaacbfsanders@REDACTED (Isaac Sanders) Date: Sat, 9 Jul 2011 09:18:59 -0400 Subject: [erlang-questions] Ways to get started Message-ID: Hello all! I am a rubyist looking for more information on erlang, and I was hoping to find some resources... If you would be so kind as to let me know any that have helped you. -- Cheers, Isaac -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Sat Jul 9 15:28:03 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sat, 9 Jul 2011 08:28:03 -0500 Subject: [erlang-questions] Inheriting functions that use record atoms Message-ID: I will have a number of modules that define a single record and functions for using that record as a model stored in a database for a webapp. There are these nice macros for converting a record to json and json to a record that will basically always look like: to_json(Record) -> ?record_to_json(?MODULE, Record). to_record(JSON) -> ?json_to_record(?MODULE, JSON). But at times they'll need to be overridden for a specific module/record. My problem is I'd like to not have to copy/paste these into every module (and they'll be more base functions like these) that is a model. extends would be best since that allows overriding but from the base module I can't reference the module that is inheriting the base module as a macro, so I can't use the json_to_record macro since I can't pass in a macro of the record atom name. A parse transform won't work either since I don't think I can add a call to a macro like I need. Does anyone have any suggestions? Or should I just not use records for this? Records are nice since I'm using mnesia and found those record_to json_to macros! Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sigmastar@REDACTED Sat Jul 9 15:50:53 2011 From: sigmastar@REDACTED (Jesse Gumm) Date: Sat, 9 Jul 2011 08:50:53 -0500 Subject: [erlang-questions] Ways to get started In-Reply-To: Message-ID: <4e185cc2.07852b0a.6cfb.ffffca72@mx.google.com> For the free approach check out www.learnyousomeerlang.com I learned from Joe Armstrong's book Programming Erlang (Pragmatic Bookshelf) There's also Erlang Programming by Francesco Cesarini (O'Reilly). Most recently is Erlang and OTP in Action, by Martin Logan (Manning). This one puts special emphasis on using the OTP framework.  I haven't read this one yet, but I've been to one of the ErlangCamps (which I gather follow a similar curriculum as the book) run by Martin, and can only imagine it's a great book. -- Jesse Gumm Sigma Star Systems 414.940.4866 On Jul 9, 2011 8:19 AM, Isaac Sanders <isaacbfsanders@REDACTED> wrote: Hello all! I am a rubyist looking for more information on erlang, and I was hoping to find some resources... If you would be so kind as to let me know any that have helped you. -- Cheers, Isaac -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Sat Jul 9 18:55:59 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Sat, 9 Jul 2011 11:55:59 -0500 Subject: [erlang-questions] Inheriting functions that use record atoms In-Reply-To: References: Message-ID: Records don't play nice with other languages. I have a similar situation that I haven't tackled yet with my mnesia store, but I solved it with my ets store by doing something like: {objtype, objid, ObjPropList, JSON} You wouldn't have to keep a copy of the JSON around, but I do because ets is my last value cache for a web server so I might as well save the serialization step. I imagine in mnesia it could be similar, or you could do something more generic -record(mdoc, {objid, objtype, obj=[], json}). % or something Your record doesn't represent your object at all, it's just something that mnesia can deal with. The real object is just a proplist, which is how JSON usually deserializes into Erlang. -Kenny On Sat, Jul 9, 2011 at 8:28 AM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > I will have a number of modules that define a single record and functions > for using that record as a model stored in a database for a webapp. > > There are these nice macros for converting a record to json and json to a > record that will basically always look like: > > to_json(Record) -> > ?record_to_json(?MODULE, Record). > > to_record(JSON) -> > ?json_to_record(?MODULE, JSON). > > But at times they'll need to be overridden for a specific module/record. > > My problem is I'd like to not have to copy/paste these into every module > (and they'll be more base functions like these) that is a model. extends > would be best since that allows overriding but from the base module I can't > reference the module that is inheriting the base module as a macro, so I > can't use the json_to_record macro since I can't pass in a macro of the > record atom name. > > A parse transform won't work either since I don't think I can add a call to > a macro like I need. > > Does anyone have any suggestions? Or should I just not use records for > this? Records are nice since I'm using mnesia and found those record_to > json_to macros! > > Thanks, > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sat Jul 9 19:39:22 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Sat, 9 Jul 2011 19:39:22 +0200 Subject: [erlang-questions] Diff algorithm in Erlang Message-ID: Is there any diff algorithm like "An O(ND) Difference Algorithm by Eugene Myers" implemented in Erlang? Or any other algorithm which generates edit script for two lists of elements? In my case those elements can be of any type which supports comparison operation (in case of Erlang, it could be anything). Yes, I've tried to find it myself, but my google fu produced no results. Best regards, Gleb Peregud From jwatte@REDACTED Sat Jul 9 20:44:39 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 9 Jul 2011 11:44:39 -0700 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: You should not send name/password in clear text. You should send it using HTTP+TLS == HTTPS! My three suggestions should be used together. The more salient point is that a REST client may or may not like cookies, because you cannot, for example, issue multiple requests in parallel until you've first done the stateful, cookie-generating, initial server round-trip. HTTPS has had some performance problems with an additional round-trip during connection negotiation for TLS. For browsers that re-use the connection (all HTTP/1.1 browser), this is not much of a problem. However, our friends at Google have actually done a lot of research in how to front-load the necessary packets, so that you can recude the negotiation overhead to close to zero, without a change in protocol! It's quite fascinating. For an initial look, check out: http://blog.chromium.org/2011/05/ssl-falsestart-performance-results.html Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Jul 8, 2011 at 10:40 PM, Max Lapshin wrote: > On Sat, Jul 9, 2011 at 9:20 AM, Jon Watte wrote: > > You've already gotten some good answers; however, having worked > extensively > > with this myself, I recommend: > > > > 1) Use HTTPS for all traffic. Computers are fast. Crypto is cheap. Cookie > > theft is a reality on open channels. > > > > This is not as clear as it seems to. > 1) HTTPS add delay. It is not a blocking problem for social > network/wiki, but it is a profit penalty for online shop > 2) HTTPS still has issues with Internet Explorer, when some code is > HTTP and some is HTTPS. And when something is HTTPS, but IE thinks it > is HTTP > > > > 2) Use Basic-auth over HTTP -- this sends name and password, > > base-64-encoded. > > > > Sorry, but it is very, very bad recommendation. You are afraid of > cookie stealing and this is why you recommend to send name and > password > plaintext on each request. What for? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mevans@REDACTED Sat Jul 9 21:15:02 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 9 Jul 2011 15:15:02 -0400 Subject: [erlang-questions] Erlang read file benchmark Message-ID: Sorry if this is a duplicate email. I can understand Erlang being a bit slower than Perl for this. Can't see an excuse for such a difference though. http://agentzh.org/#ErlangFileReadLineBenchmark Matt Sent from my iPhone From tomas.abrahamsson@REDACTED Sat Jul 9 21:55:53 2011 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Sat, 9 Jul 2011 21:55:53 +0200 Subject: [erlang-questions] Diff algorithm in Erlang In-Reply-To: References: Message-ID: > Is there any diff algorithm like "An O(ND) Difference Algorithm by > Eugene Myers" implemented in Erlang? Or any other algorithm which > generates edit script for two lists of elements? In my case those > elements can be of any type which supports comparison operation (in > case of Erlang, it could be anything). I actually had an old implementation collecting dust on the hard drive. I uploaded it to github: https://github.com/tomas-abrahamsson/tdiff in case it may be useful. BRs Tomas From mjtruog@REDACTED Sat Jul 9 22:37:51 2011 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 09 Jul 2011 13:37:51 -0700 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: Message-ID: <4E18BC1F.7000402@gmail.com> He only showed the results on the command-line. It would be nice to see results that show runtime without the startup/teardown overhead that the Erlang VM has, since it has a lot more going on than the perl interpreter. I know he briefly mentioned that the difference seemed minimal, but he posted no results to show that. On 07/09/2011 12:15 PM, Evans, Matthew wrote: > Sorry if this is a duplicate email. > > I can understand Erlang being a bit slower than Perl for this. Can't see an excuse for such a difference though. > > http://agentzh.org/#ErlangFileReadLineBenchmark > > Matt > > Sent from my iPhone > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bob@REDACTED Sun Jul 10 01:07:04 2011 From: bob@REDACTED (Bob Ippolito) Date: Sat, 9 Jul 2011 16:07:04 -0700 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: <4E18BC1F.7000402@gmail.com> References: <4E18BC1F.7000402@gmail.com> Message-ID: file:read_line does some pretty awful things, I'd expect it to be very slow. That said, there should be a much faster yet still easy way to do this quickly but there isn't one baked into OTP that I know of. On Saturday, July 9, 2011, Michael Truog wrote: > He only showed the results on the command-line. ?It would be nice to see results that show runtime without the startup/teardown overhead that the Erlang VM has, since it has a lot more going on than the perl interpreter. ?I know he briefly mentioned that the difference seemed minimal, but he posted no results to show that. > > On 07/09/2011 12:15 PM, Evans, Matthew wrote: >> Sorry if this is a duplicate email. >> >> I can understand Erlang being a bit slower than Perl for this. Can't see an excuse for such a difference though. >> >> http://agentzh.org/#ErlangFileReadLineBenchmark >> >> Matt >> >> Sent from my iPhone >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From kennethstone@REDACTED Sun Jul 10 01:08:01 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Sat, 9 Jul 2011 18:08:01 -0500 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: <4E18BC1F.7000402@gmail.com> Message-ID: Why is it awful? On Sat, Jul 9, 2011 at 6:07 PM, Bob Ippolito wrote: > file:read_line does some pretty awful things, I'd expect it to be very > slow. That said, there should be a much faster yet still easy way to > do this quickly but there isn't one baked into OTP that I know of. > > On Saturday, July 9, 2011, Michael Truog wrote: > > He only showed the results on the command-line. It would be nice to see > results that show runtime without the startup/teardown overhead that the > Erlang VM has, since it has a lot more going on than the perl interpreter. > I know he briefly mentioned that the difference seemed minimal, but he > posted no results to show that. > > > > On 07/09/2011 12:15 PM, Evans, Matthew wrote: > >> Sorry if this is a duplicate email. > >> > >> I can understand Erlang being a bit slower than Perl for this. Can't see > an excuse for such a difference though. > >> > >> http://agentzh.org/#ErlangFileReadLineBenchmark > >> > >> Matt > >> > >> Sent from my iPhone > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Jul 10 01:33:07 2011 From: bob@REDACTED (Bob Ippolito) Date: Sat, 9 Jul 2011 16:33:07 -0700 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: <4E18BC1F.7000402@gmail.com> Message-ID: I think what most people want (especially for benchmarks) is something that doesn't care about encodings and doesn't have a lot of indirection. The current solution is VERY flexible, which comes at a severe cost to performance in this case. If you read the source code you'll see that file:read_file/1 calls into io:request/2 which eventually (in another process) ends up in file_io_server:io_request/2 and ends up reading either 128 bytes or 8kb at a time, doing some unicode junk, and ends up calling io_lib:collect_line/4 to collect each line chunk at a time. If I was trying to win a benchmark I'd probably go directly to prim_file, do my own buffering, and use erlang:decode_packet/3 or the binary module to split on the newlines. If I wanted to make a nicer API I'd put that in a process to manage the buffering. On Sat, Jul 9, 2011 at 4:08 PM, Kenny Stone wrote: > Why is it awful? > > On Sat, Jul 9, 2011 at 6:07 PM, Bob Ippolito wrote: >> >> file:read_line does some pretty awful things, I'd expect it to be very >> slow. That said, there should be a much faster yet still easy way to >> do this quickly but there isn't one baked into OTP that I know of. >> >> On Saturday, July 9, 2011, Michael Truog wrote: >> > He only showed the results on the command-line. ?It would be nice to see >> > results that show runtime without the startup/teardown overhead that the >> > Erlang VM has, since it has a lot more going on than the perl interpreter. >> > ?I know he briefly mentioned that the difference seemed minimal, but he >> > posted no results to show that. >> > >> > On 07/09/2011 12:15 PM, Evans, Matthew wrote: >> >> Sorry if this is a duplicate email. >> >> >> >> I can understand Erlang being a bit slower than Perl for this. Can't >> >> see an excuse for such a difference though. >> >> >> >> http://agentzh.org/#ErlangFileReadLineBenchmark >> >> >> >> Matt >> >> >> >> Sent from my iPhone >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From mevans@REDACTED Sun Jul 10 04:43:49 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 9 Jul 2011 22:43:49 -0400 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: <4E18BC1F.7000402@gmail.com> , Message-ID: Using prim_file and the binary module produces: [mevans@REDACTED ~]$ time perl reader.pl Found 6032291 lines. real 0m1.368s user 0m1.295s sys 0m0.068s [mevans@REDACTED ~]$ time erl -noshell -s reader main Found 6048490 lines. real 0m2.090s user 0m1.965s sys 0m0.111s The Erlang code is reporting more lines because I am double counting cases where a line is split between reads. Of course, this would be easy to handle - I just wanted a proof of concept : -module(reader). -export([main/0]). -define(NL,10). main() -> {ok, File} = prim_file:open("data.log", [read,binary]), Lines = count_lines(File, 0), io:format("Found ~w lines.~n", [Lines]), halt(). count_lines(File, Count) -> case prim_file:read(File,8192) of {ok, Line} -> TC = length(binary:split(Line,[<<10>>],[global])), count_lines(File, Count+TC); _ -> Count end. ________________________________________ From: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED] On Behalf Of Bob Ippolito [bob@REDACTED] Sent: Saturday, July 09, 2011 7:33 PM To: Kenny Stone Cc: erlang-questions Questions Subject: Re: [erlang-questions] Erlang read file benchmark I think what most people want (especially for benchmarks) is something that doesn't care about encodings and doesn't have a lot of indirection. The current solution is VERY flexible, which comes at a severe cost to performance in this case. If you read the source code you'll see that file:read_file/1 calls into io:request/2 which eventually (in another process) ends up in file_io_server:io_request/2 and ends up reading either 128 bytes or 8kb at a time, doing some unicode junk, and ends up calling io_lib:collect_line/4 to collect each line chunk at a time. If I was trying to win a benchmark I'd probably go directly to prim_file, do my own buffering, and use erlang:decode_packet/3 or the binary module to split on the newlines. If I wanted to make a nicer API I'd put that in a process to manage the buffering. On Sat, Jul 9, 2011 at 4:08 PM, Kenny Stone wrote: > Why is it awful? > > On Sat, Jul 9, 2011 at 6:07 PM, Bob Ippolito wrote: >> >> file:read_line does some pretty awful things, I'd expect it to be very >> slow. That said, there should be a much faster yet still easy way to >> do this quickly but there isn't one baked into OTP that I know of. >> >> On Saturday, July 9, 2011, Michael Truog wrote: >> > He only showed the results on the command-line. It would be nice to see >> > results that show runtime without the startup/teardown overhead that the >> > Erlang VM has, since it has a lot more going on than the perl interpreter. >> > I know he briefly mentioned that the difference seemed minimal, but he >> > posted no results to show that. >> > >> > On 07/09/2011 12:15 PM, Evans, Matthew wrote: >> >> Sorry if this is a duplicate email. >> >> >> >> I can understand Erlang being a bit slower than Perl for this. Can't >> >> see an excuse for such a difference though. >> >> >> >> http://agentzh.org/#ErlangFileReadLineBenchmark >> >> >> >> Matt >> >> >> >> Sent from my iPhone >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Sun Jul 10 08:01:41 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Sun, 10 Jul 2011 09:01:41 +0300 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: <4E18BC1F.7000402@gmail.com> Message-ID: <4E194045.4070101@gmail.com> > If you read the source code you'll see that file:read_file/1 calls > into io:request/2 which eventually (in another process) ends up in > file_io_server:io_request/2 and ends up reading either 128 bytes or > 8kb at a time, doing some unicode junk, and ends up calling > io_lib:collect_line/4 to collect each line chunk at a time. > Isn't Perl interpreter going over the same or similar steps? From bob@REDACTED Sun Jul 10 08:29:56 2011 From: bob@REDACTED (Bob Ippolito) Date: Sat, 9 Jul 2011 23:29:56 -0700 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: <4E194045.4070101@gmail.com> References: <4E18BC1F.7000402@gmail.com> <4E194045.4070101@gmail.com> Message-ID: On Sat, Jul 9, 2011 at 11:01 PM, Martin Dimitrov wrote: > >> If you read the source code you'll see that file:read_file/1 calls >> into io:request/2 which eventually (in another process) ends up in >> file_io_server:io_request/2 and ends up reading either 128 bytes or >> 8kb at a time, doing some unicode junk, and ends up calling >> io_lib:collect_line/4 to collect each line chunk at a time. >> > ?Isn't Perl interpreter going over the same or similar steps? No, it's not. -bob From banibrata.dutta@REDACTED Sun Jul 10 08:52:07 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Sun, 10 Jul 2011 12:22:07 +0530 Subject: [erlang-questions] Ways to get started In-Reply-To: <4e185cc2.07852b0a.6cfb.ffffca72@mx.google.com> References: <4e185cc2.07852b0a.6cfb.ffffca72@mx.google.com> Message-ID: On Sat, Jul 9, 2011 at 7:20 PM, Jesse Gumm wrote: > For the free approach check out www.learnyousomeerlang.com > > I learned from Joe Armstrong's book Programming Erlang (Pragmatic > Bookshelf) > > There's also Erlang Programming by Francesco Cesarini (O'Reilly). > > Most recently is Erlang and OTP in Action, by Martin Logan (Manning). This > one puts special emphasis on using the OTP framework. I haven't read this > one yet, but I've been to one of the ErlangCamps (which I gather follow a > similar curriculum as the book) run by Martin, and can only imagine it's a > great book. > > Apart from that excellent suggestion, I'd like to add few more and suggested order (what worked for me): 1. First read (gives you a sense of early achievement and keeps you motivated, since you make progress quite fast) -- 'Thinking in Erlang'. 2. A kind of style guide http://www.erlang.se/doc/programming_rules.shtml 3. Joe's book (by this time, I could make progress, much faster) 4. 'Erlang in Practise' video tutorials (paid, but very economical). Much easier to follow, once you've gone thru step 1/2/3, and you can very quickly get a feel or real-life application development. While there are books which do the same, but as they say, a picture (or a video) is worth several thousand words. Coming from the Ruby (& esply so if RoR) world, you'd spoilt with choices / availability of ample such videos. While there are quite a few video tutorials for Erlang, only few are hard-core programming centric, where they show you creation of something interesting. > ------------------------------ > On Jul 9, 2011 8:19 AM, Isaac Sanders wrote: > > Hello all! I am a rubyist looking for more information on erlang, and I was > hoping to find some resources... If you would be so kind as to let me know > any that have helped you. > > -- > Cheers, > > Isaac > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From mevans@REDACTED Sun Jul 10 14:43:50 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sun, 10 Jul 2011 08:43:50 -0400 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: Message-ID: <70D46DCA-DA8B-41D5-9813-EF6FF427E356@verivue.com> I don't think anyone would disagree with the right tool for the job. My complaint is that in any sufficiently large project some form of file I/O is going to happen. If Erlang was, say 50%, slower than Perl I wouldn't care; but an order of magnitude is really off the chart. As I showed you can roll your own code using prim_file that is good enough, but most people looking at Erlang for a solution wouldn't do that. Instead they would see the linked to webpage, laugh, and never look at Erlang again. Just food for thought Matt Sent from my iPhone On Jul 10, 2011, at 2:30 AM, "Thomas Heller" wrote: > A few years ago there was a very interesting discussion on this topic > and some very smart people created some really smart Solutions. > > Google: Tim Bray and "Wide Finder" > > http://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder > http://www.tbray.org/ongoing/When/200x/2008/05/01/Wide-Finder-2 > > Conclusion was that Erlang is not the best Language for this type of > work but with some clever tricks becomes "good enough". > > http://www.duomark.com/erlang/proposals/gen_stream.html > > This has been in cooking in OTP for a while now, Jay Nelson gave a > Talk on it which you can find here: > > http://www.erlang-factory.com/conference/SFBay2011/speakers/JayNelson > > Havent used this myself. Choose the right tool for the Job, so if you > can write it in 20 lines of Perl, dont try writing it in 300 Lines of > Erlang unless you really really want/need to. > > HTH, > /thomas > > > > On Sat, Jul 9, 2011 at 9:15 PM, Evans, Matthew wrote: >> Sorry if this is a duplicate email. >> >> I can understand Erlang being a bit slower than Perl for this. Can't see an excuse for such a difference though. >> >> http://agentzh.org/#ErlangFileReadLineBenchmark >> >> Matt >> >> Sent from my iPhone >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From erlang@REDACTED Sun Jul 10 15:13:05 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 10 Jul 2011 15:13:05 +0200 Subject: [erlang-questions] Erlang read file benchmark In-Reply-To: References: <4E18BC1F.7000402@gmail.com> Message-ID: How large is the file you want to read lines from? All the files I want to process are small (ie in relation to memory) I don't think I've ever called read_line to read a file since I know its rather slow I call file:read_file read the entire file into a binary then chunk it into lines later. /Joe On Sun, Jul 10, 2011 at 1:33 AM, Bob Ippolito wrote: > I think what most people want (especially for benchmarks) is something > that doesn't care about encodings and doesn't have a lot of > indirection. The current solution is VERY flexible, which comes at a > severe cost to performance in this case. > > If you read the source code you'll see that file:read_file/1 calls > into io:request/2 which eventually (in another process) ends up in > file_io_server:io_request/2 and ends up reading either 128 bytes or > 8kb at a time, doing some unicode junk, and ends up calling > io_lib:collect_line/4 to collect each line chunk at a time. > > If I was trying to win a benchmark I'd probably go directly to > prim_file, do my own buffering, and use erlang:decode_packet/3 or the > binary module to split on the newlines. If I wanted to make a nicer > API I'd put that in a process to manage the buffering. > > On Sat, Jul 9, 2011 at 4:08 PM, Kenny Stone wrote: >> Why is it awful? >> >> On Sat, Jul 9, 2011 at 6:07 PM, Bob Ippolito wrote: >>> >>> file:read_line does some pretty awful things, I'd expect it to be very >>> slow. That said, there should be a much faster yet still easy way to >>> do this quickly but there isn't one baked into OTP that I know of. >>> >>> On Saturday, July 9, 2011, Michael Truog wrote: >>> > He only showed the results on the command-line. ?It would be nice to see >>> > results that show runtime without the startup/teardown overhead that the >>> > Erlang VM has, since it has a lot more going on than the perl interpreter. >>> > ?I know he briefly mentioned that the difference seemed minimal, but he >>> > posted no results to show that. >>> > >>> > On 07/09/2011 12:15 PM, Evans, Matthew wrote: >>> >> Sorry if this is a duplicate email. >>> >> >>> >> I can understand Erlang being a bit slower than Perl for this. Can't >>> >> see an excuse for such a difference though. >>> >> >>> >> http://agentzh.org/#ErlangFileReadLineBenchmark >>> >> >>> >> Matt >>> >> >>> >> Sent from my iPhone >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pablo.platt@REDACTED Sun Jul 10 15:46:16 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 10 Jul 2011 06:46:16 -0700 (PDT) Subject: [erlang-questions] crypto:dh_compute_key/3 requires 128 bytes keys? Message-ID: <1310305576.79371.YahooMailNeo@web112618.mail.gq1.yahoo.com> I'm trying to use crypto:dh_compute_key/3 to create a shared secret. When the OthersPublicKey length is 128 bytes (without the int32 length) it's ok but when its length is 132 I'm getting a computation_failed exception. I don't see anything in the docs that require 128 bytes length and the c++ implementation I'm trying to duplicate doesn't seem to require that. Is it correct that the equivalent to the c BN_bn2bin and BN_bin2bn functions in erlang is just to append the int32 length to the binary? Am I missing something? Please see the following example: OthersPublicKey = <<0,0,0,132,129,2,29,2,39,75,73,146,222,72,20,118,110,64,2,253,161,64,62,246, ? 27,140,79,50,38,237,13,178,255,71,226,246,68,92,250,118,182,81,224,96,149,58, ? 170,221,15,122,155,157,247,231,49,66,201,249,157,152,106,108,113,155,235,53, ? 246,83,175,57,116,223,28,97,91,83,226,139,50,27,69,219,73,138,125,86,159,208, ? 229,236,77,130,187,40,0,234,78,120,195,79,170,225,219,7,241,7,87,208,126,93, ? 4,74,1,64,242,179,222,135,241,100,96,179,135,28,195,186,147,248,20,184,33, ? 204,27,122,178,116>>, MyPrivateKey = <<0,0,0,128,91,203,79,5,193,226,103,129,180,9,151,55,246,246,58,4,27,65,61,92, ? 153,216,254,29,58,116,181,18,181,241,241,192,120,72,135,87,95,255,67,206,38, ? 68,226,146,84,103,130,15,189,36,180,184,99,245,96,81,124,80,53,230,241,18, ? 209,65,113,126,91,131,90,166,53,207,152,63,154,97,7,161,57,149,226,171,86, ? 156,203,54,4,93,245,192,153,85,81,107,24,183,42,89,81,14,188,117,109,9,170, ? 56,139,181,138,8,84,34,239,14,242,52,113,192,117,224,20,116,1,151,15,186,182, ? 140>>, DH_P = <<0,0,0,128,255,255,255,255,255,255,255,255,201,15,218,162,33,104,194,52,196, ? 198,98,139,128,220,28,209,41,2,78,8,138,103,204,116,2,11,190,166,59,19,155, ? 34,81,74,8,121,142,52,4,221,239,149,25,179,205,58,67,27,48,43,10,109,242,95, ? 20,55,79,225,53,109,109,81,194,69,228,133,181,118,98,94,126,198,244,76,66, ? 233,166,55,237,107,11,255,92,182,244,6,183,237,238,56,107,251,90,137,159,165, ? 174,159,36,17,124,75,31,230,73,40,102,81,236,230,83,129,255,255,255,255,255, ? 255,255,255>>, DH_G = <<0,0,0,1,2>>, DHParams = [DH_P, DH_G], % getting computation_failed exception crypto:dh_compute_key(OthersPublicKey, MyPrivateKey, DHParams). % remove last 4 bytes and update the length from 132 to 128 OthersPublicKey2 = <<0,0,0,128,129,2,29,2,39,75,73,146,222,72,20,118,110,64,2,253,161,64,62,246, ? 27,140,79,50,38,237,13,178,255,71,226,246,68,92,250,118,182,81,224,96,149,58, ? 170,221,15,122,155,157,247,231,49,66,201,249,157,152,106,108,113,155,235,53, ? 246,83,175,57,116,223,28,97,91,83,226,139,50,27,69,219,73,138,125,86,159,208, ? 229,236,77,130,187,40,0,234,78,120,195,79,170,225,219,7,241,7,87,208,126,93, ? 4,74,1,64,242,179,222,135,241,100,96,179,135,28,195,186,147,248,20,184,33, ? 204>>, % no exception crypto:dh_compute_key(OthersPublicKey2, MyPrivateKey, DHParam From pablo.platt@REDACTED Sun Jul 10 16:40:53 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 10 Jul 2011 07:40:53 -0700 (PDT) Subject: [erlang-questions] crypto:dh_compute_key/3 requires 128 bytes keys? In-Reply-To: <1310305576.79371.YahooMailNeo@web112618.mail.gq1.yahoo.com> References: <1310305576.79371.YahooMailNeo@web112618.mail.gq1.yahoo.com> Message-ID: <1310308853.34001.YahooMailNeo@web112615.mail.gq1.yahoo.com> I thought that the OthersPublicKey size is 132 bytes but it is actually 128 bytes. ----- Original Message ----- From: Pablo Platt To: "erlang-questions@REDACTED" Cc: Sent: Sunday, July 10, 2011 4:46 PM Subject: [erlang-questions] crypto:dh_compute_key/3 requires 128 bytes keys? I'm trying to use crypto:dh_compute_key/3 to create a shared secret. When the OthersPublicKey length is 128 bytes (without the int32 length) it's ok but when its length is 132 I'm getting a computation_failed exception. I don't see anything in the docs that require 128 bytes length and the c++ implementation I'm trying to duplicate doesn't seem to require that. Is it correct that the equivalent to the c BN_bn2bin and BN_bin2bn functions in erlang is just to append the int32 length to the binary? Am I missing something? Please see the following example: OthersPublicKey = <<0,0,0,132,129,2,29,2,39,75,73,146,222,72,20,118,110,64,2,253,161,64,62,246, ? 27,140,79,50,38,237,13,178,255,71,226,246,68,92,250,118,182,81,224,96,149,58, ? 170,221,15,122,155,157,247,231,49,66,201,249,157,152,106,108,113,155,235,53, ? 246,83,175,57,116,223,28,97,91,83,226,139,50,27,69,219,73,138,125,86,159,208, ? 229,236,77,130,187,40,0,234,78,120,195,79,170,225,219,7,241,7,87,208,126,93, ? 4,74,1,64,242,179,222,135,241,100,96,179,135,28,195,186,147,248,20,184,33, ? 204,27,122,178,116>>, MyPrivateKey = <<0,0,0,128,91,203,79,5,193,226,103,129,180,9,151,55,246,246,58,4,27,65,61,92, ? 153,216,254,29,58,116,181,18,181,241,241,192,120,72,135,87,95,255,67,206,38, ? 68,226,146,84,103,130,15,189,36,180,184,99,245,96,81,124,80,53,230,241,18, ? 209,65,113,126,91,131,90,166,53,207,152,63,154,97,7,161,57,149,226,171,86, ? 156,203,54,4,93,245,192,153,85,81,107,24,183,42,89,81,14,188,117,109,9,170, ? 56,139,181,138,8,84,34,239,14,242,52,113,192,117,224,20,116,1,151,15,186,182, ? 140>>, DH_P = <<0,0,0,128,255,255,255,255,255,255,255,255,201,15,218,162,33,104,194,52,196, ? 198,98,139,128,220,28,209,41,2,78,8,138,103,204,116,2,11,190,166,59,19,155, ? 34,81,74,8,121,142,52,4,221,239,149,25,179,205,58,67,27,48,43,10,109,242,95, ? 20,55,79,225,53,109,109,81,194,69,228,133,181,118,98,94,126,198,244,76,66, ? 233,166,55,237,107,11,255,92,182,244,6,183,237,238,56,107,251,90,137,159,165, ? 174,159,36,17,124,75,31,230,73,40,102,81,236,230,83,129,255,255,255,255,255, ? 255,255,255>>, DH_G = <<0,0,0,1,2>>, DHParams = [DH_P, DH_G], % getting computation_failed exception crypto:dh_compute_key(OthersPublicKey, MyPrivateKey, DHParams). % remove last 4 bytes and update the length from 132 to 128 OthersPublicKey2 = <<0,0,0,128,129,2,29,2,39,75,73,146,222,72,20,118,110,64,2,253,161,64,62,246, ? 27,140,79,50,38,237,13,178,255,71,226,246,68,92,250,118,182,81,224,96,149,58, ? 170,221,15,122,155,157,247,231,49,66,201,249,157,152,106,108,113,155,235,53, ? 246,83,175,57,116,223,28,97,91,83,226,139,50,27,69,219,73,138,125,86,159,208, ? 229,236,77,130,187,40,0,234,78,120,195,79,170,225,219,7,241,7,87,208,126,93, ? 4,74,1,64,242,179,222,135,241,100,96,179,135,28,195,186,147,248,20,184,33, ? 204>>, % no exception crypto:dh_compute_key(OthersPublicKey2, MyPrivateKey, DHParam _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From peralta.alejandro@REDACTED Mon Jul 11 01:21:32 2011 From: peralta.alejandro@REDACTED (Ale) Date: Sun, 10 Jul 2011 20:21:32 -0300 Subject: [erlang-questions] Asynchronous I/O - Getting documentation. In-Reply-To: References: Message-ID: Hello Ulf, Thank you very mucho for your reply, it was very helpful and informative. Regards, Ale 2011/7/8 Ulf Wiger : > > Well, IO is really synchronous from the point of view of the caller. That is, the erlang process requesting input or output is blocked while waiting for the service to be performed. In erlang terms, it performs a synchronous call (i.e. sends a message and waits for the result in a "selective receive" statement) to another erlang process. > > The request may be dispatched in different ways depending on the system. If the erlang node in question doesn't have a terminal, the request may be served transparently on another node, which does. For example, if a function is executed on a remote erlang node via an rpc:call(), any io effected by the called function will be automatically routed back to the calling node. As a result, if you call > > rpc:call(OtherNode, io, format, ["hello~n"]), > > the output, "hello", will appear in the shell of your own node - not on OtherNode. This is very useful, not least when debugging embedded systems. > > The runtime system itself either performs non-blocking IO or dispatches an OS thread to perform blocking IO. This is an implementation detail that the erlang program itself has no control over. > > BR, > Ulf W > > On 7 Jul 2011, at 04:37, Ale wrote: > >> Hello All, >> >> I'm looking for documentation to expand my knowledge of the >> implementation of how asynchronous I/O works; from wikipedia. >> >> """This approach is also used in the Erlang programming language >> runtime system. The Erlang virtual machine uses asynchronous IO using >> a small pool of only a few threads or sometimes just one process, to >> handle IO from up to millions of Erlang processes. IO handling in each >> process is written mostly using blocking synchronous I/O. This way >> high performance of asynchronous I/O is merged with simplicity of >> normal IO. Many IO problems in Erlang are mapped to message passing, >> which can be easily processed using built-in selective receive.""" >> >> I'm interested in learning how the scheduler works, and how the >> process which handle IO work. >> >> Thanks, >> -- >> Ale. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > -- Ale. From daniel@REDACTED Mon Jul 11 02:19:56 2011 From: daniel@REDACTED (Daniel Luna) Date: Mon, 11 Jul 2011 00:19:56 +0000 Subject: [erlang-questions] Thank you for a great Spawnfest Message-ID: I wish to publicly thank the Spawnfest organisers for a great programming competition. 48 hours of Erlang hacking is over. Now waiting for the results... Cheers, Daniel -- http://havebackpack.com - Exploring the world with a 30L backpack. Open to fun challenging assignments anywhere in the world. From sigmastar@REDACTED Mon Jul 11 03:11:58 2011 From: sigmastar@REDACTED (Jesse Gumm) Date: Sun, 10 Jul 2011 20:11:58 -0500 Subject: [erlang-questions] Thank you for a great Spawnfest In-Reply-To: Message-ID: <4e1a4de5.502be70a.56a1.65e8@mx.google.com> Agreed! That was a ton of fun and a bit of stress. -Jesse -- Jesse Gumm Sigma Star Systems 414.940.4866 On Jul 10, 2011 7:20 PM, Daniel Luna <daniel@REDACTED> wrote: I wish to publicly thank the Spawnfest organisers for a great programming competition. 48 hours of Erlang hacking is over. Now waiting for the results... Cheers, Daniel -- http://havebackpack.com - Exploring the world with a 30L backpack. Open to fun challenging assignments anywhere in the world. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jul 11 02:13:34 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jul 2011 12:13:34 +1200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> Message-ID: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> On 9/07/2011, at 2:44 AM, Ulf Wiger wrote: > Having said this, I invite anyone who goes through that kind of exercise to share their results. Not only will it help you evaluate the experiment honestly; it will increase the store of experiments that can be copied and tailored to the specific challenges of the next project. I had an unpleasant experimental experience of my own last year. Let me first give you the lesson I learned, and then the background. LESSON: Expect your experiment to surprise you, probably by showing the experiment was a waste of time. Background: I'm sick of arguments about style. To my mind, it is so obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I cannotUnderstandWhyOtherPeopleDoNotSeeThat. But they don't. My zeroth, the New Zealand Anglican Church has even brought out an electronic version of the liturgy called WePray, in a desperate attempt to seem hip. (Since it is only available for an operating system sold by what may be the largest software company to have been convicted to software piracy, I wonder what their ethics committee were doing. But I digress.) So I devised a little language called Chatterton (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), which allowed me to mechanically produce several style variants of some sample programs and ask some 3rd year software engineering students to find some mistakes in them. What I expected was one of three things: - no measurable effect - more readable code (i.e., NOT baStudlyCaps) being easier to fix - more familiar (i.e., JustLikeXingJava) being easier to fix. What I *got* was students telling me they couldn't read code on paper; they needed syntax-colouring IDEs (the listings all fitted on a single sheet of paper and used black-and-white styling) and ideally a debugger so they could find mistakes by stepping through the code. I also got students telling me that it was horribly unreasonable of me to expect them to read a 30-page manual; NOBODY could read that much. And finding a definition of an identifier in a 2-page listing is just beyond human capacity; it's impossible to do that without the machine assistance of an IDE. So the whole experiment produced no worthwhile data for reasons having nothing to do with what I thought I was testing. As other people have been saying, the "Erlang web servers challenge" is at serious risk of producing no worthwhile data. From ttmrichter@REDACTED Mon Jul 11 04:39:51 2011 From: ttmrichter@REDACTED (Michael Richter) Date: Mon, 11 Jul 2011 10:39:51 +0800 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: On 11 July 2011 08:13, Richard O'Keefe wrote: > What I *got* was students telling me they couldn't read code on > paper; they needed syntax-colouring IDEs (the listings all fitted > on a single sheet of paper and used black-and-white styling) and > ideally a debugger so they could find mistakes by stepping through > the code. I also got students telling me that it was horribly > unreasonable of me to expect them to read a 30-page manual; NOBODY > could read that much. And finding a definition of an identifier > in a 2-page listing is just beyond human capacity; it's impossible > to do that without the machine assistance of an IDE. > > So the whole experiment produced no worthwhile data for reasons having > nothing to do with what I thought I was testing. > I strongly disagree, Mr. Fleming. You got some very worthwhile data about just how poorly-educated, poorly-motivated and maybe even lazy students are in your neck of the woods. It's not the data you set out to find with your experiment, but it remains very important, worthwhile, even valuable data. Many important discoveries have happened in experiments that set out to find something entirely different. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jul 11 05:00:26 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jul 2011 15:00:26 +1200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal Message-ID: Matthew Sackman has a presentation http://erlang-factory.com/upload/presentations/435/MatthewSackman.pdf on extending Erlang syntax to make it a nicer functional language. I'd like to briefly discuss his "cut" proposal. First, some background. Pop-2 had the idea of "frozen formals". You could provide a function with some of its arguments, like this: f(% foo, bar %) What you got was a closure that pushed the "frozen formals" onto the stack after any other parameters and then jumped to the original function. It was not quite the same as fun (X, Y) -> f(X, Y, Foo, Bar) end because the frozen formals were evaluated at the time the closure was created, not the time it was called. In functional languages where functions notionally have one argument, you get the same effect by passing the leading arguments but not all of them. Since f X Y Z = E is equivalent to f = fun (X) -> fun (Y) -> fun (Z) -> E end end end in those languages, passing just a few of the arguments is what notionally happens anyway in a "saturated" function call, so no extra syntax is needed. Erlang is _not_ "Curried" like SML or Haskell, not even notionally. Possibly the nearest analogue for Erlang would be Eiffel (ECMA-367), where agent signifies a closure, with '?' indicating an "open" (unfilled) argument, and other arguments indicating "closed" (filled) ones, evaluated when the 'agent' expression is evaluated, not when it is called. Without the aid of extra combinators, Pop-2 can only pre-supply trailing arguments, Haskell can only pre-supply leading arguments, and Eiffel can pre-supply any mix of arguments. Matthew Sackman's proposal is very close to Eiffel agent expressions. He uses '_' instead of '_' and 'cut(...)' instead of 'agent ...', but it's basically the same idea. I may say here that the word 'cut' makes absolutely no sense to me. Let's look at his first three examples: % Current Erlang info_all(VHostPath, Items) -> map(VHostPath, fun (Q) -> info(Q, Items) end). % With "cut" info_all(VHostPath, Items) -> map(VHostPath, info(_, Items)). % or info_all(VHostPath, Items) -> map(VHostPath, cut(info(_, Items))). % Current Erlang again: info_all(V_Host_Path, Items) -> [info(Q, Items) || Q <- V_Host_Path]. % Current Erlang backing_queue_timeout(State = #q{backing_queue = BQ}) -> run_backing_queue(BQ, fun (M, BQS) -> M:timeout(BQS) end, State). % With "cut" backing_queue_timeout(State = #q{backing_queue = BQ}) -> run_backing_queue(BQ, _:timeout(_), State). % or backing_queue_timeout(State = #q{backing_queue = BQ}) -> run_backing_queue(BQ, cut(_:timeout(_)), State). And here's just what I learned to hate in Eiffel: the correspondence between place-holders and parameters is positional. Suppose I want fun (BQS, M) -> M:timeout(BSQ) end then I am out of luck. You _can't_ do that with "_". % Current Erlang reset_msg_expiry_fun(TTL) -> fun (MsgProps) -> MsgProps#message_properties{expiry = calculate_msg_expiry(TTL)} end. % With "cut" reset_msg_expiry_fun(TTL) -> _#message_properties{expiry = calculate_msg_expiry(TTL)}. % or reset_msg_expiry_fun(TTL) -> cut(_#message_properties{expiry = calculate_msg_expiry(TTL)}). A reminder here that there is a difference between a 'fun' equivalent in other languages and a 'cut' equivalent: in the 'fun' equivalent the other arguments are evaluated when the closure is *called*, while in the 'cut' equivalent the other arguments are evaluated when the closure is *defined*. If calculate_msg_expiry/1 has side effects, the two functions of reset_msg_expiry_fun/1 are not (if "cut" is implemented for least surprise) equivalent. Now let's look at some other funs: AC = spawn_link(fun() -> init(Init, KernelApp) end), Without the "cut" keyword, "_" is no help because you would not be able to tell init(Init, KernelApp) -- the call from init(Init, KernelApp) -- the closure So it's clear that the "cut" keyword (or rather, something meaningful to play the same r?le) is necessary if we want to do AC = spawn_link(cut(init(Init, KernelApp))). ets:filter(ac_tab, fun([{{loaded, AppName}, #appl{descr = Descr, vsn = Vsn}}]) -> {true, {AppName, Descr, Vsn}}; (_) -> false end, []) "_" cannot help here because - there are two clauses - one of them involves pattern matching. map(fun([Key, Val]) -> {Key, Val} end, ) "_" cannot help here because of pattern matching. lists:foreach(fun(Appl) -> ets:insert(ac_tab, {{loaded, Appl#appl.name}, Appl}) end, NewAppls), This could be done better as [ets:insert(ac_tab, {{loaded, Appl#appl.name}, Appl}) || Appl <- NewAppls] fun ({Name, Id}) -> case Id of ... "_" cannot help here because the body is not a simple function call. The argument involves pattern matching. If it were fun (Name, Id) -> case Id of ... we'd still be stuck because Name isn't the first parameter to appear in the body. I began a survey of 'fun's in the kernel/ directory. Out of the first 22, one could use "_". I have to go and pick up my daughter from school, so I don't have time to finish this survey yet. However, it doesn't look like that much of a win so far. From overminddl1@REDACTED Mon Jul 11 05:19:10 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 10 Jul 2011 21:19:10 -0600 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: I hope that is a joke. I inhaled 1200 page manuals like popcorn when I was a student 20 years ago, and still do to this day. I *hope* that being lazy is their only problem. Yeesh... On Jul 10, 2011 7:33 PM, "Richard O'Keefe" wrote: > > On 9/07/2011, at 2:44 AM, Ulf Wiger wrote: >> Having said this, I invite anyone who goes through that kind of exercise to share their results. Not only will it help you evaluate the experiment honestly; it will increase the store of experiments that can be copied and tailored to the specific challenges of the next project. > > I had an unpleasant experimental experience of my own last year. > Let me first give you the lesson I learned, and then the background. > > LESSON: Expect your experiment to surprise you, > probably by showing the experiment was a waste of time. > > Background: I'm sick of arguments about style. To my mind, it is so > obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I > cannotUnderstandWhyOtherPeopleDoNotSeeThat. But they don't. My zeroth, > the New Zealand Anglican Church has even brought out an electronic > version of the liturgy called WePray, in a desperate attempt to seem > hip. (Since it is only available for an operating system sold by what > may be the largest software company to have been convicted to software > piracy, I wonder what their ethics committee were doing. But I digress.) > > So I devised a little language called Chatterton > (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), > which allowed me to mechanically produce several style variants of > some sample programs and ask some 3rd year software engineering students > to find some mistakes in them. > > What I expected was one of three things: > - no measurable effect > - more readable code (i.e., NOT baStudlyCaps) being easier to fix > - more familiar (i.e., JustLikeXingJava) being easier to fix. > > What I *got* was students telling me they couldn't read code on > paper; they needed syntax-colouring IDEs (the listings all fitted > on a single sheet of paper and used black-and-white styling) and > ideally a debugger so they could find mistakes by stepping through > the code. I also got students telling me that it was horribly > unreasonable of me to expect them to read a 30-page manual; NOBODY > could read that much. And finding a definition of an identifier > in a 2-page listing is just beyond human capacity; it's impossible > to do that without the machine assistance of an IDE. > > So the whole experiment produced no worthwhile data for reasons having > nothing to do with what I thought I was testing. > > As other people have been saying, the "Erlang web servers challenge" > is at serious risk of producing no worthwhile data. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Mon Jul 11 05:45:35 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 10 Jul 2011 21:45:35 -0600 Subject: [erlang-questions] Matthew Sackman's "cut" proposal Message-ID: Typing this from phone, so forgive any terseness. I have used his Erlang cut library, simplified code and made it more readable, but the limitations above are pretty nasty. I recommend a style copying that of Boost.Phoenix, a C++ library designed for heavy functional usage, specifically it allows for what cut does, but also reordering of parameters and skipping parameters (as well as much more that goes well beyond the score of Erlang). It does it like this (in an Erlangish way): fun(Arg1, _Arg2, Arg3) -> Arg1:aCall(Arg3), Arg3 end (Arg1:aCall("test", Arg3), Arg3) Probably use _ instead of Arg as the tag, or perhaps language support for something else that is not sorted by the current grammar. Phoenix only goes up to Arg9 by default I think, but you can specify all others by templates: arg_<12>() I do not really care what the tag is, but if it is followed by its positional number, that fixes a lot. P.S. The reason that Pheonix starts at Arg1 instead of Arg0 is because Arg0 is a tuple of all the args as I recall, excellent for debugging. P.P.S Pheonix seems dynamically typed because it is. It is a template tree that is evaluated, compiled, and even has optimization passes performed on it (in many cases outperforming hand written code to do the same thing, at worse matching it) by a template 'compiler', Boost.Proto, at the point that it is called. Ingenious design. Hefty compile time. ;-) On Jul 10, 2011 9:00 PM, "Richard O'Keefe" wrote: -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jul 11 07:34:11 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jul 2011 17:34:11 +1200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: Message-ID: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> On 11/07/2011, at 3:00 PM, Richard O'Keefe wrote: > > I began a survey of 'fun's in the kernel/ directory. > Out of the first 22, one could use "_". Here are the counts. funs may be ruled out for multiple reasons; in each such case only the most severe reason was counted. 46 the fun should be replaced by a comprehension 30 the fun has multiple clauses 72 the fun has a complex body, a 'case' or 'if' or a comma or an operator. 22 the fun does pattern matching on one or more arguments 4 the fun wants its arguments in a different order from the underlying function call 7 one or more arguments is in a "deep" position 26 the fun has no arguments (this is usually in a spawn of some kind, or possibly a transaction) 27 funs are candidates for the "cut" "_" treatment. However, I didn't bother distinguishing, until it was too late, between funs that could have been fun foo/N and the rest; about half of these candidates should be of that form. So about 1 'fun' out of 14 or more could use the Matthew Sackman treatment. This seems to depend on style; for a long time I thought there weren't going to be _any_ candidates but they were clumped in a small number of files. Look for example at f_send = fun(S,D) -> inet_tcp:send(S,D) end, f_recv = fun(S,N,T) -> inet_tcp:recv(S,N,T) end, f_setopts_pre_nodeup = fun(S) -> inet:setopts(S, [{active, false}, {packet, 4}, nodelay()]) end, f_setopts_post_nodeup = fun(S) -> inet:setopts(S, [{active, true}, {deliver, port}, {packet, 4}, nodelay()]) end, f_getll = fun(S) -> inet:getll(S) end, which could have been f_send = fun inet_tcp:send/2, f_recv = fun inet_tcp:recv/3, f_setops_pre_nodeup = cut(inet:setopts(_, [{active,false},{packet,4},nodelay()]), f_setopts_post_nodeup = cut(inet:setopts(_, [{active,true},{deliver,port},{packet,4},nodelay()]), f_getll = fun inet:getll/1 We could just add setopts(Opts) -> fun (Socket) -> prim_inet:setopts(Socket, Opts) end. or better still, nodeup_opts(Opts) -> fun (Socket) -> prim_inet:setopts(Socket, [{packet,4},nodelay()]++Opts) end. and then we'd have f_send = fun inet_tcp:send/2, f_recv = fun inet_tcp:recv/3, f_setops_pre_nodeup = nodeup_opts([{active,false}]), f_setopts_post_nodeup = nodeop_opts([{active,true},{deliver,port}]), f_getll = fun inet:getll/1 and there'd be no need for the "cut" "_" treatment at all. Why did Pop-2 need partial application? Because it used dynamic scope for variables, so that returning a lambda would not have worked. Why does Eiffel have agents? Because it didn't have lambdas. Erlang _does_ have working funs so we can do manual Currying on the rare occasions when it's needed. From ok@REDACTED Mon Jul 11 07:37:02 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jul 2011 17:37:02 +1200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: <23FDF476-1BEF-4F5B-8B77-1A29EDA288E2@cs.otago.ac.nz> On 11/07/2011, at 3:19 PM, OvermindDL1 wrote: > I hope that is a joke. I inhaled 1200 page manuals like popcorn when I was a student 20 years ago, and still do to this day. I *hope* that being lazy is their only problem. Yeesh... I wish it were a joke. And to be fair, not all of the students complained of this. But about 1/4 of the students did complain of that. And a similar number complained of basically being unable to read 2-page programs without a syntax-colouring IDE. From josh@REDACTED Mon Jul 11 07:43:30 2011 From: josh@REDACTED (Josh Johnston) Date: Mon, 11 Jul 2011 15:43:30 +1000 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <23FDF476-1BEF-4F5B-8B77-1A29EDA288E2@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> <23FDF476-1BEF-4F5B-8B77-1A29EDA288E2@cs.otago.ac.nz> Message-ID: Sad truth in my experience is that this isn't limited to students, as you'll see most clearly when you try to hire for a junior position :P There seems to be a lot of people going under the title of programmer who are actually "IDE users" or "framework configurers" or "API implementers", and when you ask them to do some actual programming you can see how lost they are in it. On 11/07/2011, at 3:37 PM, Richard O'Keefe wrote: > > On 11/07/2011, at 3:19 PM, OvermindDL1 wrote: > >> I hope that is a joke. I inhaled 1200 page manuals like popcorn when I was a student 20 years ago, and still do to this day. I *hope* that being lazy is their only problem. Yeesh... > > I wish it were a joke. And to be fair, not all of the students complained of this. > But about 1/4 of the students did complain of that. > And a similar number complained of basically being unable to read 2-page > programs without a syntax-colouring IDE. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Mon Jul 11 07:49:58 2011 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 11 Jul 2011 17:49:58 +1200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> <23FDF476-1BEF-4F5B-8B77-1A29EDA288E2@cs.otago.ac.nz> Message-ID: <97644CD9-CB55-45DB-84EB-F9E244227E9C@cs.otago.ac.nz> On 11/07/2011, at 5:43 PM, Josh Johnston wrote: > Sad truth in my experience is that this isn't limited to students, as you'll see most clearly when you try to hire for a junior position :P There seems to be a lot of people going under the title of programmer who are actually "IDE users" or "framework configurers" or "API implementers", and when you ask them to do some actual programming you can see how lost they are in it. Someone from another country applied to do a research degree here. The CV listed "C, C++, Java" under "skills". I wrote back asking "tell me about the last C program you wrote. What did you think you did well? What would you do differently next time?" and the response was "well, I only did a course on it, but I'm a fast learner, so I'm sure I'll be expert after a couple of weeks." I'm going to use that question again (:-). (My suspicions about this student's Java skills were raised by a claim to have used 'mavan' a lot...) From ryan.volpe@REDACTED Mon Jul 11 08:31:38 2011 From: ryan.volpe@REDACTED (Ryan Volpe) Date: Mon, 11 Jul 2011 02:31:38 -0400 Subject: [erlang-questions] Code loaders other than efile and inet Message-ID: Greetings all; Sorry for the lengthy post -- TL;DR, see bottom. I asked a rather condensed version of this on the #erlang IRC channel last week; the consensus was that it should be asked here, so I put together some further notes. I previously attempted (last week) to send this message to this list to no avail, so forgive me if it somehow does wind up as a double post. *Overview* I'm not sure if it's something I'm missing, but the docs seem to suggest that custom code loaders are supported by the standard ERTS (from erl_prim_loader(3) ): The -loader Loader command line flag can be used to choose the method used > by the erl_prim_loader. Two Loader methods are supported by the Erlang > runtime system: efile and inet. If another loader is required, then it has > to be implemented by the user. The Loader provided by the user must > fulfill the protocol defined below, and it is started with the > erl_prim_loader by evaluating open_port({spawn,Loader},[binary]). ? The following protocol must be followed if a user provided loader port > program is used. The Loader port program is started with the command > open_port({spawn,Loader},[binary]). The protocol is as follows: Function Send Receive ------------------------------------------------------------- get_file [102 | FileName] [121 | BinaryFile] (on success) [122] (failure) stop eof terminate *Trial Run* So far, so good, right? Looks like there's a fairly well-defined means of implementing a code loader port. So let's try it? *annabel:loaders ryan$* *cat test_loader.c* > #include > #include > #include > #include #define BUFSIZE 10 * 1024 > extern int errno; int main(int argc, char * const argv[]) > { > char req; > char readbuf[BUFSIZE]; > char fname[BUFSIZE + 1]; > ssize_t nread; > > fprintf(stderr, "**%s:: received %d arguments\n", argv[0], argc-1); > for (int i=1; i { > fprintf(stderr, "**%s:: argument %d: %s\n", argv[0], i-1, argv[i]); > } > > while (1) > { > /* use unbuffered I/O [read/write(2) v. fread/fwrite(3)] */ > nread = read(0, (void *)&req, sizeof(char)); > if (nread == 0) > { > fprintf(stderr, "**%s:: stdin at eof, terminating...\n", > argv[0]); > exit(0); > } else if (nread < 0) { > perror("*** error reading port request"); > exit(-errno); > } else if (req != 'f') { > fprintf(stderr, "**%s:: unknown mode ('%c') requested\n", > argv[0], req); > write(1, (void *)"z", sizeof(char)); > } else { > nread = read(0, (void *)&readbuf, sizeof(char) * BUFSIZE); > strncpy(fname, readbuf, nread + 1); > fprintf(stderr, "**%s:get_file(\"%s\")\n", argv[0], fname); > write(1, (void *)"z", sizeof(char)); > } > } > > return 0; > } > *annabel:loaders ryan$* *cat test_loader.erl* > -module (test_loader). > -export ([get_file/1]). > -export ([start/0, start/1, interact/1]). -spec get_file(string() | atom()) -> binary() | failure. > -spec start() -> atom(). > -spec interact(binary()) -> {char(), binary()}. get_file(Filename) when is_atom(Filename) -> > get_file(atom_to_list(Filename)); > get_file(Filename) -> > Req = list_to_binary([$f|Filename]), > {Flag, BinaryFile} = interact(Req), > case Flag of > $z -> failure; > $y -> BinaryFile > end. start() -> sta rt(test_loader). > start(Loader) -> > Port = open_port({spawn, Loader}, [binary]), > register(test_loader, Port). interact(Req) -> > case whereis(test_loader) of > undefined -> > start(), > interact(Req); > Port -> > Port ! {self(), {command, Req}}, > receive > {Port, {data, <>}} -> {Flag, Binary} > end > end. > *annabel:loaders ryan$* *CFLAGS='-std=c99 -Wall -pedantic' make > test_loader* > cc -std=c99 -Wall -pedantic test_loader.c -o test_loader > *annabel:loaders ryan$* *PATH=.:$PATH erl* > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Eshell V5.8.4 (abort with ^G) > 1> c(test_loader). > {ok,test_loader} > 2> test_loader:start(). > true > 3> **test_loader:: received 0 arguments 3> test_loader:get_file("testing"). > **test_loader:get_file("testing") > failure > 4> > User switch command > --> q > **test_loader:: stdin at eof, terminating... *annabel:loaders ryan$* *PATH=.:$PATH erl -loader test_loader* {"cannot start > loader",{function_clause,[{erl_prim_loader,start_it,["test_loader",none,<0.2.0>,[]]}]}} {"init terminating in > do_boot",{function_clause,[{erl_prim_loader,start_it,["test_loader",none,<0.2.0>,[]]}]}} (no error logger present) error: "Error in process <0.3.0> with exit value: > {function_clause,[{erl_prim_loader,start_it,[\"test_loader\",none,<0.2.0>,[]]}]}\n" > Crash dump was written to: erl_crash.dump init terminating in do_boot () *Error Location* Referring to erl_prim_loader ($OTP_SRC/erts/preloaded/erl_prim_loader.erl), the error reported soon reveals itself in start_it/4 (presumably where the open_port/2 call referenced in erl_prim_loader(3) would be made): start_it("ose_inet"=Cmd, Id, Pid, Hosts) -> > ? > start_it("inet", Id, Pid, Hosts); > start_it("inet", Id, Pid, Hosts) -> > ? > loop(State, Pid, []); > start_it("efile", Id, Pid, _Hosts) -> > ? > loop(State, Pid, []). *Issue 1: *No clauses are defined here for custom code loaders, only "ose_inet", "inet" and "efile". *Further Issues* Continuing down the file, further issues crop up. erl_prim_loader:loop/3 relies on a corpus of helper functions (all defined in the same module) to do its dirty work; several of these have issues. *Issue 2:* Several functions only accept efile or inet for #state.loader -- any other value will cause a function_clause error: - handle_get_file/3 - handle_list_dir/2 - handle_get_cwd/2 - handle_stop/2 - handle_exit/3 - handle_timeout/2 *Issue 3:* Two other functions only accept efile for #state.loader -- same error as above: - handle_set_primary_archive/4 - handle_release_archives/1 *Issue 4:* handle_get_files/4 will fail for any loader besides efile that sets #state.multi_get to true. *TL;DR Summary* * * I can't see how this shipped for so many versions as "broken" as it seems -- I can't have been the first one to attempt to implement a custom code loader. Have I just entirely missed the boat on how this is done, or are these actually bugs? Most importantly, is there any interest in a patch that fixes these apparent issues? * * Regards, Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Jul 11 11:15:44 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 11 Jul 2011 10:15:44 +0100 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: On 11 July 2011 06:34, Richard O'Keefe wrote: > > Why did Pop-2 need partial application? > Because it used dynamic scope for variables, so that returning a lambda > would not have worked. > Why does Eiffel have agents? > Because it didn't have lambdas. > > Erlang _does_ have working funs so we can do manual Currying on the rare > occasions when it's needed. > For me, the compelling reason to have *something* like this is that function composition and partial application are the bread and butter of my programming style, and in Erlang they both prove to be rather, what shall we say, wordy? For example, consider this function (a hamcrest matcher) which isn't too bad at all, but is indicative of the amount of noise we have to deal with: -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). ends_with(X) -> fun(Y) -> string:equal(string:right(Y, length(X)), X) end. Not personally, I would much sooner have the compiler deal with the anonymous function machinery for me: -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). ends_with(X) -> string:equal(string:right(?, length(X)), X) end. Note that I've substituted '_' for ? in order to get away from discussing the syntax - I could happily go with any marker-character because to be perfectly honest I don't care that much.It would be even better if I could express things in point free style, a la Haskell. In this simple example, typing in the 'fun(...) ->' part isn't too onerous, but there are plenty of times when it's annoying. The other point about Mathew's work, is that it provides a means to eliminate trillions (ahem) of intermediate (state) variables in a section of code such as: doodle(X) -> Result1 = do_thing(X), Result2 = calculate_next_thing(Result1), Result3 = get_bored_of_typing_result_x(Result2), Result4 = start_getting_annoyed(Result3), %% etc, etc, etc {ok, Result23}. Yes I know that "explaining variables" are better than silly and meaningless naming conventions such as Result_N, but the point is that I'd like to *inline* this code, but can't easily do so without readability suffering. Being able to chain/pipeline function calls would be *really* nice, so I could rewrite this in a neater style: Fun = quite_happy/1 + not_so_bored/1 + calculate_next_thing/1 + do_thing/1, ... I can't understand why the parser requires me to type "fun M:F/A" when the "/" character clearly indicates that I'm referencing a function in a special way. And if the compiler can recognise a reference to a function (local or otherwise), then why not have it recognise "+" as an operator for chaining function calls? Anyways - I think this work is interesting and hope that it sparks a good conversation about how we can potentially improve support for function composition and (potentially) partial application in Erlang. From gleber.p@REDACTED Mon Jul 11 11:26:20 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 11 Jul 2011 11:26:20 +0200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: On Mon, Jul 11, 2011 at 07:34, Richard O'Keefe wrote: > Here are the counts. ?funs may be ruled out for multiple > reasons; in each such case only the most severe reason was > counted. > [snip] Just a quick idea. Will _1, _2, _3, etc. placeholders work for positional currying? I'm guessing that it might be possible to implement it in Matt's cut parse transform. From erlang@REDACTED Mon Jul 11 12:54:12 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 11 Jul 2011 12:54:12 +0200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: On Mon, Jul 11, 2011 at 2:13 AM, Richard O'Keefe wrote: > > On 9/07/2011, at 2:44 AM, Ulf Wiger wrote: >> Having said this, I invite anyone who goes through that kind of exercise to share their results. Not only will it help you evaluate the experiment honestly; it will increase the store of experiments that can be copied and tailored to the specific challenges of the next project. > > I had an unpleasant experimental experience of my own last year. > Let me first give you the lesson I learned, and then the background. > > LESSON: Expect your experiment to surprise you, > ? ? ? ?probably by showing the experiment was a waste of time. > > Background: I'm sick of arguments about style. ?To my mind, it is so > obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I > cannotUnderstandWhyOtherPeopleDoNotSeeThat. ?But they don't. ?My zeroth, > the New Zealand Anglican Church has even brought out an electronic > version of the liturgy called WePray, in a desperate attempt to seem > hip. ?(Since it is only available for an operating system sold by what > may be the largest software company to have been convicted to software > piracy, I wonder what their ethics committee were doing. ?But I digress.) > > So I devised a little language called Chatterton > (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), > which allowed me to mechanically produce several style variants of > some sample programs and ask some 3rd year software engineering students > to find some mistakes in them. > > What I expected was one of three things: > ?- no measurable effect > ?- more readable code (i.e., NOT baStudlyCaps) being easier to fix > ?- more familiar (i.e., JustLikeXingJava) being easier to fix. > > What I *got* was students telling me they couldn't read code on > paper; they needed syntax-colouring IDEs (the listings all fitted > on a single sheet of paper and used black-and-white styling) and > ideally a debugger so they could find mistakes by stepping through > the code. ?I also got students telling me that it was horribly > unreasonable of me to expect them to read a 30-page manual; NOBODY > could read that much. Oh dear - Attention deficiency syndrome - I suspect, though cannot prove, that is is a consequence of our brains being constantly bombarded with irrelevant information. The cure is simple: You set reading them off Proust for a few hours a day with no interruptions. (for example: from Cities of the Plain (Sodom et Gomorrhe) [Vol. 4 of Remembrance of Things Past-- (? la Recherche du temps perdu)]) "Their honour precarious, their liberty provisional, lasting only until the discovery of their crime; their position unstable, like that of the poet who one day was feasted at every table, applauded in every theatre in London, and on the next was driven from every lodging, unable to find a pillow upon which to lay his head, turning the mill like Samson and saying like him: "The two sexes shall die, each in a place apart!"; excluded even, save on the days of general disaster when the majority rally round the victim as the Jews rallied round Dreyfus, from the sympathy--at times from the society--of their fellows, in whom they inspire only disgust at seeing themselves as they are, portrayed in a mirror which, ceasing to flatter them, accentuates every blemish that they have refused to observe in themselves, and makes them understand that what they have been calling their love (a thing to which, playing upon the word, they have by association annexed all that poetry, painting, music, chivalry, asceticism have contrived to add to love) springs not from an ideal of beauty which they have chosen but from an incurable malady; like the Jews again (save some who will associate only with others of their race and have always on their lips ritual words and consecrated pleasantries), shunning one another, seeking out those who are most directly their opposite, who do not desire their company, pardoning their rebuffs, moved to ecstasy by their condescension; but also brought into the company of their own kind by the ostracism that strikes them, the opprobrium under which they have fallen, having finally been invested, by a persecution similar to that of Israel, with the physical and moral characteristics of a race, sometimes beautiful, often hideous, finding (in spite of all the mockery with which he who, more closely blended with, better assimilated to the opposing race, is relatively, in appearance, the least inverted, heaps upon him who has remained more so) a relief in frequenting the society of their kind, and even some corroboration of their own life, so much so that, while steadfastly denying that they are a race (the name of which is the vilest of insults), those who succeed in concealing the fact that they belong to it they readily unmask, with a view less to injuring them, though they have no scruple about that, than to excusing themselves; and, going in search (as a doctor seeks cases of appendicitis) of cases of inversion in history, taking pleasure in recalling that Socrates was one of themselves, as the Israelites claim that Jesus was one of them, without reflecting that there were no abnormals when homosexuality was the norm, no anti-Christians before Christ, that the disgrace alone makes the crime because it has allowed to survive only those who remained obdurate to every warning, to every example, to every punishment, by virtue of an innate disposition so peculiar that it is more repugnant to other men (even though it may be accompanied by exalted moral qualities) than certain other vices which exclude those qualities, such as theft, cruelty, breach of faith, vices better understood and so more readily excused by the generality of men; forming a freemasonry far more extensive, more powerful and less suspected than that of the Lodges, for it rests upon an identity of tastes, needs, habits, dangers, apprenticeship, knowledge, traffic, glossary, and one in which the members themselves, who intend not to know one another, recognise one another immediately by natural or conventional, involuntary or deliberate signs which indicate one of his congeners to the beggar in the street, in the great nobleman whose carriage door he is shutting, to the father in the suitor for his daughter's hand, to him who has sought healing, absolution, defence, in the doctor, the priest, the barrister to whom he has had recourse; all of them obliged to protect their own secret but having their part in a secret shared with the others, which the rest of humanity does not suspect and which means that to them the most wildly improbable tales of adventure seem true, for in this romantic, anachronistic life the ambassador is a bosom friend of the felon, the prince, with a certain independence of action with which his aristocratic breeding has furnished him, and which the trembling little cit would lack, on leaving the duchess's party goes off to confer in private with the hooligan; a reprobate part of the human whole, but an important part, suspected where it does not exist, flaunting itself, insolent and unpunished, where its existence is never guessed; numbering its adherents everywhere, among the people, in the army, in the church, in the prison, on the throne; living, in short, at least to a great extent, in a playful and perilous intimacy with the men of the other race, provoking them, playing with them by speaking of its vice as of something alien to it; a game that is rendered easy by the blindness or duplicity of the others, a game that may be kept up for years until the day of the scandal, on which these lion-tamers are devoured; until then, obliged to make a secret of their lives, to turn away their eyes from the things on which they would naturally fasten them, to fasten them upon those from which they would naturally turn away, to change the gender of many of the words in their vocabulary, a social constraint, slight in comparison with the inward constraint which their vice, or what is improperly so called, imposes upon them with regard not so much now to others as to themselves, and in such a way that to themselves it does not appear a vice." One they have got used to reading sentences like this that ramble on for a few pages without getting anywhere in particular they will find a 30-page manual easy going. /Joe >?And finding a definition of an identifier > in a 2-page listing is just beyond human capacity; it's impossible > to do that without the machine assistance of an IDE. > > So the whole experiment produced no worthwhile data for reasons having > nothing to do with what I thought I was testing. > > As other people have been saying, the "Erlang web servers challenge" > is at serious risk of producing no worthwhile data. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From banibrata.dutta@REDACTED Mon Jul 11 14:39:34 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 11 Jul 2011 18:09:34 +0530 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: On Mon, Jul 11, 2011 at 8:49 AM, OvermindDL1 wrote: > I hope that is a joke. I inhaled 1200 page manuals like popcorn when I was > a student 20 years ago, and still do to this day. I *hope* that being lazy > is their only problem. Yeesh... > There are people who can do cross-country, then some who can do only sprints and the rest just love to watch it as a spectator sports (worst case, on TV). Devouring (or inhaling) 1200 page manual, are characteristics of a really gifted stud-reader. Our profession has a huge number of engineers who are definitely above average (as per some standard :-) ) but not as gifted. I am afraid, if you let them devour the 1200 page manual over a year, they might still not have gone past the first 120pages or so. Reading through 1200 pages, and retaining everything requires not only the gift of amazing retention, but that of focus, patience and resolve. Being one of the lesser mortals, I still yearn for precooked information in form of easy-to-grasp video tutorials, which is why I dug Kevin Smith's "Erlang in Practise" so much. Also, there is a difference in approach. I know quite a few very good (I'd say much better / smarter than me) engineers, who refer to the manual (or reference material) "Just in Time". Given so much happening around, so much to see, so much to learn, often times, it works. However, I do agree, that when it comes to academic rigour, it is probabaly a general observation that Gen-X has it easier than Gen-Y, although not always. > On Jul 10, 2011 7:33 PM, "Richard O'Keefe" wrote: > > > > On 9/07/2011, at 2:44 AM, Ulf Wiger wrote: > >> Having said this, I invite anyone who goes through that kind of exercise > to share their results. Not only will it help you evaluate the experiment > honestly; it will increase the store of experiments that can be copied and > tailored to the specific challenges of the next project. > > > > I had an unpleasant experimental experience of my own last year. > > Let me first give you the lesson I learned, and then the background. > > > > LESSON: Expect your experiment to surprise you, > > probably by showing the experiment was a waste of time. > > > > Background: I'm sick of arguments about style. To my mind, it is so > > obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I > > cannotUnderstandWhyOtherPeopleDoNotSeeThat. But they don't. My zeroth, > > the New Zealand Anglican Church has even brought out an electronic > > version of the liturgy called WePray, in a desperate attempt to seem > > hip. (Since it is only available for an operating system sold by what > > may be the largest software company to have been convicted to software > > piracy, I wonder what their ethics committee were doing. But I digress.) > > > > So I devised a little language called Chatterton > > (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), > > which allowed me to mechanically produce several style variants of > > some sample programs and ask some 3rd year software engineering students > > to find some mistakes in them. > > > > What I expected was one of three things: > > - no measurable effect > > - more readable code (i.e., NOT baStudlyCaps) being easier to fix > > - more familiar (i.e., JustLikeXingJava) being easier to fix. > > > > What I *got* was students telling me they couldn't read code on > > paper; they needed syntax-colouring IDEs (the listings all fitted > > on a single sheet of paper and used black-and-white styling) and > > ideally a debugger so they could find mistakes by stepping through > > the code. I also got students telling me that it was horribly > > unreasonable of me to expect them to read a 30-page manual; NOBODY > > could read that much. And finding a definition of an identifier > > in a 2-page listing is just beyond human capacity; it's impossible > > to do that without the machine assistance of an IDE. > > > > So the whole experiment produced no worthwhile data for reasons having > > nothing to do with what I thought I was testing. > > > > As other people have been saying, the "Erlang web servers challenge" > > is at serious risk of producing no worthwhile data. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pkplex@REDACTED Mon Jul 11 15:00:17 2011 From: pkplex@REDACTED (Joshua Sandbrook) Date: Tue, 12 Jul 2011 01:00:17 +1200 Subject: [erlang-questions] Problem with wxErlang, distel, emacs, on windows. Message-ID: Hello. While using distel in emacs on windows to run something simple like wx:demo()., it hangs (as in the demo is not displayed... I can type in the erlang console, nothing comes back. Only thing it responds to is C-c C-c to kill it. From some googling, the only hint I have is that it is some sort of stdio contest going on between wxErlang and emacs/distel. Any ideas? Running R14B03 on windows 7. Thanks, Josh. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Jul 11 17:54:56 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 11 Jul 2011 09:54:56 -0600 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: On Sun, Jul 10, 2011 at 6:13 PM, Richard O'Keefe wrote: > > So I devised a little language called Chatterton > (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), > which allowed me to mechanically produce several style variants of > some sample programs and ask some 3rd year software engineering students > to find some mistakes in them. > > What I expected was one of three things: > ?- no measurable effect > ?- more readable code (i.e., NOT baStudlyCaps) being easier to fix > ?- more familiar (i.e., JustLikeXingJava) being easier to fix. > > What I *got* was students telling me they couldn't read code on > paper; they needed syntax-colouring IDEs (the listings all fitted > on a single sheet of paper and used black-and-white styling) and > ideally a debugger so they could find mistakes by stepping through > the code. Do the students have a consistent background - e.g. a common intro course? E.g. if schools are teaching programming using Java, I could see an IDE culture emerging. Considering the huge amount of pointless code that Java requires you to write, I can't really blame students for getting hooked on visual aids. Other languages may apply. My uni used Pascal, which isn't as onerous to write and read as Java. But we had debuggers and someone somewhere encouraged me to routinely use the debugger to step through code to "reason about it". Since then I've found functional languages better suited to this process. Is the answer, "I have no idea what's wrong here -- let me step through the code" more a product of imperative languages than to laziness? Garrett From essen@REDACTED Mon Jul 11 18:01:47 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 11 Jul 2011 18:01:47 +0200 Subject: [erlang-questions] Thank you for a great Spawnfest In-Reply-To: References: Message-ID: <4E1B1E6B.9050103@dev-extend.eu> On 07/11/2011 02:19 AM, Daniel Luna wrote: > I wish to publicly thank the Spawnfest organisers for a great > programming competition. > > 48 hours of Erlang hacking is over. Now waiting for the results... Most importantly thanks to everyone who participated! It looks like there was some great new open source projects being worked on this week-end. I'm sure they'll greatly benefit the community. Great work, everyone! -- Lo?c Hoguin Dev:Extend From g@REDACTED Mon Jul 11 18:59:09 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 11 Jul 2011 10:59:09 -0600 Subject: [erlang-questions] Ways to get started In-Reply-To: References: Message-ID: On Sat, Jul 9, 2011 at 7:18 AM, Isaac Sanders wrote: > Hello all! I am a rubyist looking for more information on erlang, and I was > hoping to find some resources... If you would be so kind as to let me know > any that have helped you. In addition to the books and the online resources mentioned... I find it helps to pick a project that you'd be comfortable solving in another language, then tackle it in Erlang. Solve mini problems as you go. Ask questions here. I find http://erldocs.com/ helpful as a reference. Garrett P.S. Come to think of it, I don't know if http://www.erlang.org/doc.html was mentioned. None of the books are a substitute for that. From g@REDACTED Mon Jul 11 19:16:03 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 11 Jul 2011 11:16:03 -0600 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: On Fri, Jul 8, 2011 at 11:20 PM, Jon Watte wrote: > 2) Use Basic-auth over HTTP -- this sends name and password, > base-64-encoded. This is surely a typo. You can't say "HTTP" and expect people to read "HTTP + TLS". For simple web auth, I routinely use basic auth, but only ever over HTTPS. This doesn't work however if you need to control sessions or let users log out. It's just a quick and dirty way to control who can see what. Garrett From ulf.wiger@REDACTED Mon Jul 11 19:27:05 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 11 Jul 2011 19:27:05 +0200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: <1D242B7D-C86F-4640-9975-E81305331058@erlang-solutions.com> On 11 Jul 2011, at 02:13, Richard O'Keefe wrote: > What I *got* was students telling me they couldn't read code on > paper; they needed syntax-colouring IDEs (the listings all fitted > on a single sheet of paper and used black-and-white styling) and > ideally a debugger so they could find mistakes by stepping through > the code. I also got students telling me that it was horribly > unreasonable of me to expect them to read a 30-page manual; NOBODY > could read that much. And finding a definition of an identifier > in a 2-page listing is just beyond human capacity; it's impossible > to do that without the machine assistance of an IDE. I'm reminded of a lecture in Programming Parallel Systems (by Jens Riboe) at KTH, where a student stopped him after having scribbled a code example on the blackboard: - What's that? - ??? - That code you just wrote - what language is it? - That? It's just pseudo-code - Well, we only know Pascal! - Pseudo-code is not an actual language, it's just a sketch of a program. - But we haven't been taught pseudo-code; we know only Pascal. - But you can't express this neatly in Pascal? - That's the only language we've been taught. Thanks for sharing your experience. :) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From pmacgown@REDACTED Mon Jul 11 19:41:09 2011 From: pmacgown@REDACTED (pmacgown@REDACTED) Date: Mon, 11 Jul 2011 17:41:09 +0000 (UTC) Subject: [erlang-questions] Ways to get started In-Reply-To: Message-ID: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> The biggest problem that I am running into is that my Unix sys admin experience 10+ years ago doesn't help much.? Using Git, Rebar, and even make is a steep hill for me to travel.? I've been through numerous tutorials and documentation and there is this massive body of knowledge that I just don't have, or these tutorials just assume you know .? I get Erlang. That's the easy part.? It's just the underlying development structure and environment that gets in my way. Is there a real 101 that can give me a leg up?? Really basic questions like: How do I get Erlang from github?? Where do I put it once I get it?? How does rebar fit into this picture?? What does a development cycle look like using these tools? Really Really basic stuff. --Peter ----- Original Message ----- From: "Garrett Smith" To: "Isaac Sanders" Cc: erlang-questions@REDACTED Sent: Monday, July 11, 2011 12:59:09 PM Subject: Re: [erlang-questions] Ways to get started On Sat, Jul 9, 2011 at 7:18 AM, Isaac Sanders wrote: > Hello all! I am a rubyist looking for more information on erlang, and I was > hoping to find some resources... If you would be so kind as to let me know > any that have helped you. In addition to the books and the online resources mentioned... I find it helps to pick a project that you'd be comfortable solving in another language, then tackle it in Erlang. Solve mini problems as you go. Ask questions here. I find http://erldocs.com/ helpful as a reference. Garrett P.S. Come to think of it, I don't know if http://www.erlang.org/doc.html was mentioned. None of the books are a substitute for that. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Jul 11 20:13:30 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 11 Jul 2011 12:13:30 -0600 Subject: [erlang-questions] Ways to get started In-Reply-To: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Mon, Jul 11, 2011 at 11:41 AM, wrote: > I get Erlang. That's the easy part.? It's just the underlying development > structure and environment that gets in my way. I hit the same issue. I went so far as to pick up Emacs. I have to say, I'm quite happy with the result, but that's a pretty big pill to swallow when you want to simply add Erlang to your repertoire. And you don't *have* to use Emacs. But for me it was the path of least resistance. I'd be curious to know what editor/IDE/toolset people use other than Emacs. Catch is, you have to be overall happy with it :) > Is there a real 101 that can give me a leg up?? Really basic questions like: > > How do I get Erlang from github?? Where do I put it once I get it? Use a system package (e.g. aptitude, yum, pacman) or a mainstream installer. I wouldn't mess with compiling from scratch, much less getting source from github. > How does rebar fit into this picture? Rebar is emerging as the de facto standard for building Erlang projects. Most projects that use rebar also provide a Makefile, but I wouldn't worry about make files in your case. There are a few good "getting started" guides with rebar. I'd start with those. > What does a development cycle look like using these tools? The most basic work flow is to use an editor to modify your Erlang source files and then run your compilation ("rebar compile") in a separate shell. It gets more streamlined from there depending on the toolset. Garrett From banibrata.dutta@REDACTED Mon Jul 11 20:52:42 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Tue, 12 Jul 2011 00:22:42 +0530 Subject: [erlang-questions] Ways to get started In-Reply-To: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Mon, Jul 11, 2011 at 11:11 PM, wrote: > The biggest problem that I am running into is that my Unix sys admin > experience 10+ years ago doesn't help much. Using Git, Rebar, and even make > is a steep hill for me to travel. I've been through numerous tutorials and > documentation and there is this massive body of knowledge that I just don't > have, or these tutorials just assume you know. > > > > I get Erlang. That's the easy part. It's just the underlying development > structure and environment that gets in my way. > > Is there a real 101 that can give me a leg up? Really basic questions > like: > > > > How do I get Erlang from github? Where do I put it once I get it? How > does rebar fit into this picture? What does a development cycle look like > using these tools? > > Peter, What you describe, I think, is quite typical of people who are picking up a new language, and if they've not been programming very actively, lately. I am in that stage, and have gone thru similar struggles. However, in the end, one need to master each of those tools, to learn Erlang. In fact, I think it is pointless to master rebar and git, if you are just a beginner. What might suffice (to tide over the curve), is to know a very small handful of tool command vocabulary. For example -- 1) cloning a git repository 2) setting up a rebar project 3) compiling/building project with rebar rest can wait. I've fallen into the Emacs honey pot (every one, in every video turtorial/presentation on Erlang, seems to be using Emacs), and struggled to learn just enough, although, now I realize that with Erlide (Emace based) or even with vi + command-line bash aliases, I am as productive. > > > Really Really basic stuff. > > > > --Peter > > ------------------------------ > > *From: *"Garrett Smith" > *To: *"Isaac Sanders" > *Cc: *erlang-questions@REDACTED > *Sent: *Monday, July 11, 2011 12:59:09 PM > *Subject: *Re: [erlang-questions] Ways to get started > > > On Sat, Jul 9, 2011 at 7:18 AM, Isaac Sanders > wrote: > > Hello all! I am a rubyist looking for more information on erlang, and I > was > > hoping to find some resources... If you would be so kind as to let me > know > > any that have helped you. > > In addition to the books and the online resources mentioned... > > I find it helps to pick a project that you'd be comfortable solving in > another language, then tackle it in Erlang. Solve mini problems as you > go. Ask questions here. > > I find http://erldocs.com/ helpful as a reference. > > Garrett > > P.S. Come to think of it, I don't know if > http://www.erlang.org/doc.html was mentioned. None of the books are a > substitute for that. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Mon Jul 11 22:27:15 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 11 Jul 2011 16:27:15 -0400 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Jul 11, 2011, at 2:13 PM, Garrett Smith wrote: > > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) I use Text Mate, it's fast and there's a decent Erlang bundle available. CLI for building and everything else. > > Use a system package (e.g. aptitude, yum, pacman) or a mainstream > installer. I wouldn't mess with compiling from scratch, much less > getting source from github. I always build from the official sources. It's quite easy, basically just configure, make, sudo make install Distro packages tend to be outdated, Mac packagers as well. Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Jul 11 22:49:29 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 11 Jul 2011 22:49:29 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: I'm old school - you don't need any fancy tools. Just a text edit and an erlang shell. Open your text editor: Type in the following program. -module(hello). -compile(export_all). start() -> "hello world". store it in a file called hello.erl Start and erlang shell. It will say ">" Type in two commands: > c(hello). > hello:start(). The first command compiles the program. The second evaluates the command hello:start() Thats all it takes - typing three lines of code into a file with a text editor - then typing two lines into the shell. That's all it takes. 95% of all the fun can be achieved with a simple text editor and the erlang shell. That's how most of the erlang system was implemented. The erlang shell can be installed in zillions of ways - compile the sources or apt-get install it (or whatever) Forget about git/IDEs/rebar etc. Use this approach for all languages. IDEs and build tools are the single biggest obstacle I know of to getting started. Me I use - a shell - makefiles - emacs for all know programming languages under the sun. 98% of all the fun can be had with the compiler alone - all the rest is hype. Forget about the tools - Tools like rebar etc are under to automate something but if you don't know what it is that you are automating and if the tool doesn't work you will just end up being incredible confused. Then buy a decent book and type in the programs by hand. One at a time thinking as you go. After 30 years you will get the hang of this and be a good programmer. Tools are no substitute for typing in small programs and understanding exactly how they work. This is true for all programming language. Programming is an art form, there is no easy way. Like playing the violin - is there an easy way of playing the violin other than by practising for thousands of hours? I think not. Start with one liners in the shell, start the shell: type > A = 1 then > A = 2 ask what happens and why. There is no quick way to learn programming - no tool will help. Your brain is a zillion times better than the best IDE. programs form in you brain not in an IDE. But then I'm old school. Have fun - if it's not fun it is pointless - don't fight the tools all you need is a text editor and the erlang shell to start with. /Joe On Mon, Jul 11, 2011 at 10:27 PM, Mihai Balea wrote: > > On Jul 11, 2011, at 2:13 PM, Garrett Smith wrote: > > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) > > I use Text Mate, it's fast and there's a decent Erlang bundle available. > CLI for building and everything else. > > > Use a system package (e.g. aptitude, yum, pacman) or a mainstream > installer. I wouldn't mess with compiling from scratch, much less > getting source from github. > > I always build from the official sources. It's quite easy, basically just > configure, make, sudo make install > Distro packages tend to be outdated, Mac packagers as well. > > Mihai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From skruger@REDACTED Mon Jul 11 22:55:38 2011 From: skruger@REDACTED (Shaun Kruger) Date: Mon, 11 Jul 2011 14:55:38 -0600 (MDT) Subject: [erlang-questions] Ways to get started In-Reply-To: Message-ID: > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) I have to say that I enjoy erlide in eclipse. It is nice when I'm doing extensive application development. I particularly like the compile/load on save functionality it provides. I have had to play with different versions to get the performance I wanted, but things have gotten better recently. I still log to external files and tail the logs to watch for supervisor reports. I completely developed Surrogate (https://github.com/skruger/Surrogate) using ErlIDE and it has been a very enjoyable experience. Shaun ----- Original Message ----- > From: "Garrett Smith" > To: pmacgown@REDACTED > Cc: erlang-questions@REDACTED > Sent: Monday, July 11, 2011 12:13:30 PM > Subject: Re: [erlang-questions] Ways to get started > > On Mon, Jul 11, 2011 at 11:41 AM, wrote: > > I get Erlang. That's the easy part.? It's just the underlying > > development > > structure and environment that gets in my way. > > I hit the same issue. I went so far as to pick up Emacs. I have to > say, I'm quite happy with the result, but that's a pretty big pill to > swallow when you want to simply add Erlang to your repertoire. > > And you don't *have* to use Emacs. But for me it was the path of > least > resistance. > > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) > > > Is there a real 101 that can give me a leg up?? Really basic > > questions like: > > > > How do I get Erlang from github?? Where do I put it once I get it? > > Use a system package (e.g. aptitude, yum, pacman) or a mainstream > installer. I wouldn't mess with compiling from scratch, much less > getting source from github. > > > How does rebar fit into this picture? > > Rebar is emerging as the de facto standard for building Erlang > projects. Most projects that use rebar also provide a Makefile, but I > wouldn't worry about make files in your case. There are a few good > "getting started" guides with rebar. I'd start with those. > > > What does a development cycle look like using these tools? > > The most basic work flow is to use an editor to modify your Erlang > source files and then run your compilation ("rebar compile") in a > separate shell. It gets more streamlined from there depending on the > toolset. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From freza@REDACTED Mon Jul 11 23:22:11 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 11 Jul 2011 22:22:11 +0100 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: <20110711212211.GA992@hanele.lan> # Tim Watson 2011-07-11: > The other point about Mathew's work, is that it provides a means to > eliminate trillions (ahem) of intermediate (state) variables in a > section of code such as: > > doodle(X) -> > Result1 = do_thing(X), > Result2 = calculate_next_thing(Result1), > Result3 = get_bored_of_typing_result_x(Result2), > Result4 = start_getting_annoyed(Result3), > %% etc, etc, etc > {ok, Result23}. Right, that's pretty silly indeed, so why not write: doodle(X) -> {ok, compose([fun do_thing/1, fun calculate_next_thing/1, fun get_bored_of_typing_result_x/1, fun start_getting_annoyed/1], X)}. instead (where compose/2 has the obvious definition). Except the only really silly thing here is the unreasonable function names, because otherwise you'd just write something like: doodle(X) -> {ok, foo(bar(baz(bam(qux(X)))))}. What was the problem again? Do you have a more substantial example? BR, -- Jachym From sigmastar@REDACTED Mon Jul 11 23:53:25 2011 From: sigmastar@REDACTED (Jesse Gumm) Date: Mon, 11 Jul 2011 16:53:25 -0500 Subject: [erlang-questions] Ways to get started In-Reply-To: Message-ID: <4e1b70da.a8c8ec0a.55f3.0de3@mx.google.com> I'm a Gnu Screen (byobu version) split vertically and vim guy. I'm perfectly happy with my setup. It's simple, portable and gets the job done. -Jesse > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) Garrett _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh@REDACTED Tue Jul 12 00:03:38 2011 From: josh@REDACTED (Josh Johnston) Date: Tue, 12 Jul 2011 08:03:38 +1000 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: <508CD709-75C1-452F-A644-C5402B43E83A@x-team.com> I'd like to add to this and say that when I started out with erlang, Makefiles were one of the hurdles I faced. Having never written a makefile before it seemed an unnecessarily cryptic way to get files compiled. Of course, there were plenty of examples out there and I was able to _mostly_ get it to do what I wanted. But like Joe describes below, it doesn't need to be like that. My epiphany came the day that I realized what my makefile was doing: stepping through .erl files, and using erlc to create .beam files. No magic. With that realization it took me 5 minutes and a few lines of php to do _exactly_ what I wanted and never have to bend my head around a makefile again. Those who have spent years writing makefiles might find this a little stupid :P But to me it removed the hurdle and made it easy to get stuff done, which is why we write code, right? :) It seems that complicated tools and processes can often hide the underlying simplicity. Of course you might get to a point where you need those tools, but for a beginner the most important thing is to dispel the illusion and see how simple it actually can be. Josh. On 12/07/2011, at 6:49 AM, Joe Armstrong wrote: > I'm old school - you don't need any fancy tools. Just a text > edit and an erlang shell. > > Open your text editor: > > Type in the following program. > > -module(hello). > -compile(export_all). > start() -> "hello world". > > store it in a file called hello.erl > > Start and erlang shell. It will say ">" > Type in two commands: > >> c(hello). >> hello:start(). > > The first command compiles the program. The second evaluates > the command hello:start() > > Thats all it takes - typing three lines of code into a file > with a text editor - then typing two lines into the shell. > > That's all it takes. 95% of all the fun can be achieved with a simple > text editor and the erlang shell. That's how most of the erlang system > was implemented. > > The erlang shell can be installed in zillions of ways - compile the > sources or apt-get install it (or whatever) > > Forget about git/IDEs/rebar etc. > > Use this approach for all languages. > > IDEs and build tools are the single biggest obstacle I know > of to getting started. > > Me I use > > - a shell > - makefiles > - emacs > > for all know programming languages under the sun. > > 98% of all the fun can be had with the compiler alone - all the rest > is hype. > > Forget about the tools - > > Tools like rebar etc are under to automate something but if you don't > know what it is that you are automating and if the tool doesn't work > you will just end up being incredible confused. > > Then buy a decent book and type in the programs by hand. > One at a time thinking as you go. > > After 30 years you will get the hang of this and be a good programmer. > > Tools are no substitute for typing in small programs and > understanding exactly how they work. This is true for all programming > language. Programming is an art form, there is no easy way. > Like playing the violin - is there an easy way of playing the violin other > than by practising for thousands of hours? I think not. > > Start with one liners in the shell, start the shell: > > type >> A = 1 > > then > >> A = 2 > > ask what happens and why. > > There is no quick way to learn programming - no tool will help. > > Your brain is a zillion times better than the best IDE. programs > form in you brain not in an IDE. > > But then I'm old school. > > Have fun - if it's not fun it is pointless - don't fight the tools > all you need is a text editor and the erlang shell to start with. > > /Joe > > > > > > > > > > On Mon, Jul 11, 2011 at 10:27 PM, Mihai Balea wrote: >> >> On Jul 11, 2011, at 2:13 PM, Garrett Smith wrote: >> >> I'd be curious to know what editor/IDE/toolset people use other than >> Emacs. Catch is, you have to be overall happy with it :) >> >> I use Text Mate, it's fast and there's a decent Erlang bundle available. >> CLI for building and everything else. >> >> >> Use a system package (e.g. aptitude, yum, pacman) or a mainstream >> installer. I wouldn't mess with compiling from scratch, much less >> getting source from github. >> >> I always build from the official sources. It's quite easy, basically just >> configure, make, sudo make install >> Distro packages tend to be outdated, Mac packagers as well. >> >> Mihai >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dan@REDACTED Tue Jul 12 00:14:49 2011 From: dan@REDACTED (Daniel Dormont) Date: Mon, 11 Jul 2011 18:14:49 -0400 Subject: [erlang-questions] Erlang-Java interface through ordinary ports Message-ID: Hello Erlangers, I would like to have an Erlang application (specifically the auth_external module in Ejabberd) communicate with a Java program I am writing. Not as a node, but rather using the port driver API, since that is what Ejabberd already supports. From the information I've seen, it shouldn't be too terribly hard - just listen for data in System.in, look for a two-byte chunk that when read as a short specifies the number of additional bytes to read, and repeat. I'm just wondering if this is something that's considered normal and reasonable to do, any gotchas I should be aware of, etc. The only reason I'm picking Java is the rest of my application is Java and I'd like to reuse some libraries and such. thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Tue Jul 12 00:17:15 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Mon, 11 Jul 2011 15:17:15 -0700 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Mon, Jul 11, 2011 at 11:13 AM, Garrett Smith wrote: > I'd be curious to know what editor/IDE/toolset people use other than > Emacs. Catch is, you have to be overall happy with it :) joe, with a syntax highlighting file I made myself. Looks like this: https://lh3.googleusercontent.com/-oCPlbMVnPe8/Te7Puuzu5jI/AAAAAAAABnE/w-56jRE_tSo/s800/erlsyntaxinjoe.PNG (pay no heed to the ugly random bits of code, this was early in my experimenting) That, plus a bash shell in another window, and an instance of werl. From ok@REDACTED Tue Jul 12 00:59:24 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 12 Jul 2011 10:59:24 +1200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: On 11/07/2011, at 9:15 PM, Tim Watson wrote: > On 11 July 2011 06:34, Richard O'Keefe wrote: >> >> Why did Pop-2 need partial application? >> Because it used dynamic scope for variables, so that returning a lambda >> would not have worked. >> Why does Eiffel have agents? >> Because it didn't have lambdas. >> >> Erlang _does_ have working funs so we can do manual Currying on the rare >> occasions when it's needed. >> > > For me, the compelling reason to have *something* like this is that > function composition and partial application are the bread and butter > of my programming style, This is not unlike saying "assignment and goto are the bread and buffer of my C style, so Erlang should support assignment and goto". Function composition and partial application are not part of >>Erlang's<< native style. Again, the old joke: Doctor: What's the problem? Patient: It hurts when I do ? Doctor: Then don't do . If you try to write Erlang as if it were SML (or CML), it's obviously going to hurt. So the answer is don't do that. > and in Erlang they both prove to be rather, > what shall we say, wordy? For example, consider this function (a > hamcrest matcher) which isn't too bad at all, but is indicative of the > amount of noise we have to deal with: > > -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). > ends_with(X) -> > fun(Y) -> string:equal(string:right(Y, length(X)), X) end. The amount of "noise" here is precisely three tokens: 'fun' '->' 'end'. Your suffering cannot be said to be extreme. As an occasional Haskell and SML programmer, I am certainly not against Curried functions. What I *am* against is an approach which is ugly, confusing, and almost never applicable. If Erlang syntax were generalised to ends_with(X)(Y) -> string:equal(string:right(Y, length(X)), X). we could eliminate those three tokens. What's more, we could do that *WITHOUT* any weird restrictions on - whether pattern matching is allowed in the fun - the form of the fun body - where the arguments appear in the fun body - the order in which the arguments appear in the fun body. and *WITHOUT* - any weird non-intention-revealing tokens, whether '_' or '?' > Not personally, I would much sooner have the compiler deal with the > anonymous function machinery for me: > > -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). > ends_with(X) -> > string:equal(string:right(?, length(X)), X) end. It would be much much better as ends_with(Suffix)(String) -> string:equal(string:right(String, length(Suffix)), Suffix). > > Note that I've substituted '_' for ? in order to get away from No, you substituted '?' for '_', or replaced '_' with '?'. > discussing the syntax - I could happily go with any marker-character My argument is that ANY marker character is wrong because - you can't nest closures that way (does this ? belong to the inner fun or the outer one?) - you can't read the intent of a parameter from a weirdo like '?' the way you can from a name like String - you have all those other weird restrictions I mentioned above If you want to do Haskell-like things, it doesn't make sense not to do it in a Haskell-like way, and that is to say that ()()...() -> is equivalent to () -> fun () -> ... fun () -> end ... end *NO* new tokens! *NO* weird restrictions! Good consistency with the way Clean and ML and Haskell do it. *NO* changes elsewhere in the language. > The other point about Mathew's work, is that it provides a means to > eliminate trillions (ahem) Ahem indeed. > of intermediate (state) variables in a > section of code such as: > > doodle(X) -> > Result1 = do_thing(X), > Result2 = calculate_next_thing(Result1), > Result3 = get_bored_of_typing_result_x(Result2), > Result4 = start_getting_annoyed(Result3), > %% etc, etc, etc > {ok, Result23}. > > Yes I know that "explaining variables" are better than silly and > meaningless naming conventions such as Result_N, but the point is that > I'd like to *inline* this code, but can't easily do so without > readability suffering. Being able to chain/pipeline function calls > would be *really* nice, so I could rewrite this in a neater style: > > Fun = quite_happy/1 + not_so_bored/1 + calculate_next_thing/1 + do_thing/1, ... Let me see you do that with more than one argument to each function. Let's see what we have to do now: c(F1, F2) -> fun (X) -> F2(F1(X)) end. c(F1, F2, F3) -> fun (X) -> F3(F2(F1(X))) end. c(F1, F2, F3, F4) -> fun (X) -> F4(F3(F2(F1(X)))) end. doodle(X) -> F = c(fun do_thing/1, fun calculate_next_thing/1, fun get_bored_of_typing_result_x, fun start_getting_annoyed_with_lazy_programmers/1), {ok, F(X)}. Nope, not hard. Trivial, in fact. > > I can't understand why the parser requires me to type "fun M:F/A" when > the "/" character clearly indicates that I'm referencing a function in > a special way. The question is, in a language without static types, how do you tell the difference between an intentional foo/2 meaning a function and an erroneous foo/2 that should have been Foo/2, with Foo a variable bound to a number. What _I_ don't understand is why fun F/2 is not allowed with F a variable. > And if the compiler can recognise a reference to a > function (local or otherwise), then why not have it recognise "+" as > an operator for chaining function calls? Well, I've done enough mathematics to be driven screaming with rage up the nearest handy wall at any use of "+" for an operation that is not commutative. The operation you want here is often called ";" in denotational semantics. It's (F;G) X = G(F(X)). Again, in general the compiler *CANNOT* recognise a reference to a function. Haskell and SML manage that by means of type checking, and *they* don't overload function composition with any other operation. > > Anyways - I think this work is interesting and hope that it sparks a > good conversation about how we can potentially improve support for > function composition and (potentially) partial application in Erlang. The Sackman proposal does NOTHING to help with function composition, and there are, as I've shown, much better ways to do partial application. From magnus.falk@REDACTED Tue Jul 12 01:00:23 2011 From: magnus.falk@REDACTED (Magnus Falk) Date: Tue, 12 Jul 2011 01:00:23 +0200 Subject: [erlang-questions] Examining Erlang crash dumps - how to account for all memory? In-Reply-To: References: Message-ID: Turns out that the problem was twofold: The heap is listen in words not bytes and I forgot to convert, also I shouldn't have counted the unused portions of the heap since they are already included in the other numbers. So all the memory is now accounted for, now I just have to figure out why the usage explodes like that... Cheers, Magnus On Sat, Jul 9, 2011 at 7:14 AM, Jon Watte wrote: > Certain objects, such as binaries and ets tables, live in different memory > than that of each process. If you have lots of those, then that might > explain the rest of the memory. > Also, if you look at what each process is doing, is any process "garbing"? > If so, it probably is using more memory temporarily for the new block it's > compacting into. > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Fri, Jul 8, 2011 at 7:28 AM, Magnus Falk wrote: > >> This is a copy of a question I asked over at StackOverflow that noone has >> been able to answer yet: >> http://stackoverflow.com/questions/6616101/examining-erlang-crash-dumps-how-to-account-for-all-memory >> >> I've been poring over this Erlang crash dump where the VM has run out of >> heap memory. The problem is that there is no obvious culprit allocating all >> that memory. >> >> Using some serious black awk magic I've summed up the fields Stack+heap, >> OldHeap, Heap unused and OldHeap unused for each process and ranked them by >> memory usage. The problem is that this number doesn't come even close to the >> number that is representing the total memory for all the processes >> processes_used according to the Erlang crash dump guide. >> >> I've already tried the Crashdump Viewer and either I'm missing something >> or there isn't much help there for my kind of problem. >> >> The number I get is 525 MB whereas the processes_used value is at 1348 MB. >> Where can I find the rest of the memory? >> >> Cheers, >> Magnus >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jul 12 01:00:39 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 12 Jul 2011 11:00:39 +1200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: <96975BC9-501E-4CB2-98B8-536EACB7F5E1@cs.otago.ac.nz> On 11/07/2011, at 9:26 PM, Gleb Peregud wrote: > On Mon, Jul 11, 2011 at 07:34, Richard O'Keefe wrote: >> Here are the counts. funs may be ruled out for multiple >> reasons; in each such case only the most severe reason was >> counted. >> [snip] > > Just a quick idea. Will _1, _2, _3, etc. placeholders work for > positional currying? I'm guessing that it might be possible to > implement it in Matt's cut parse transform. But "arguments in the wrong order" was a *very* rare reason for the Eiffel 'agents' approach being inapplicable. It adds extra unreadable complexity while giving almost no benefit whatever. Why put Band-Aids on the Titanic? From thaterlangguy@REDACTED Tue Jul 12 01:03:15 2011 From: thaterlangguy@REDACTED (Ahmed Al-Saadi) Date: Mon, 11 Jul 2011 19:03:15 -0400 Subject: [erlang-questions] Ways to get started In-Reply-To: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: <4E1B8133.2010804@gmail.com> Peter: The thing is, for any significant project, you *will* need a development tool chain either to reduce complexity, automate tasks, or simply because someone else is using a tool in their project which you'd like to build. Here, I use the term "tool chain" loosely to describe all tools that you *could* use to develop software, not in the strict compiler tool chain sense. These typically include (not an exhaustive list): * Source Control: Git, Mercurial, Subversion, ... * Build Managment/Automation: Rebar, GNU Make, Buildout, Rake, CMake, SCons, ... * Text Editor: vim, emacs, textmate, ... * Preprocessors/Compilers: erlc, gcc, javac * Virtual Machine/Runtime: erl vm, CRuby, CPython, java vm The last three are all you need to use Erlang. But the first two are critical for any serious project. Maybe what you are looking for is for someone to guide you through an example development environment, complete with using a tool from each of the categories above. However, it's rarely that there is a default toolset for any language and developers/engineers will use what they like (and will change often). I use Mercurial/Git, Make, Vim among other tools. I suggest you look up the basic use of each tool separately. Sure it takes time, but you don't have to master each to be productive. For example, I don't have to understand the three different types of Make variable assignments to compile a package by typing: ./configure; make; sudo make install. Cheers, -signed(ahmed). On 7/11/11 1:41 PM, pmacgown@REDACTED wrote: > > The biggest problem that I am running into is that my Unix sys admin > experience 10+ years ago doesn't help much. Using Git, Rebar, and > even make is a steep hill for me to travel. I've been through > numerous tutorials and documentation and there is this massive body of > knowledge that I just don't have, or these tutorials just assume you > know. > > I get Erlang. That's the easy part. It's just the underlying > development structure and environment that gets in my way. > > Is there a real 101 that can give me a leg up? Really basic questions > like: > > How do I get Erlang from github? Where do I put it once I get it? > How does rebar fit into this picture? What does a development cycle > look like using these tools? > > Really Really basic stuff. > > --Peter > > ------------------------------------------------------------------------ > > *From: *"Garrett Smith" > *To: *"Isaac Sanders" > *Cc: *erlang-questions@REDACTED > *Sent: *Monday, July 11, 2011 12:59:09 PM > *Subject: *Re: [erlang-questions] Ways to get started > > On Sat, Jul 9, 2011 at 7:18 AM, Isaac Sanders > wrote: > > Hello all! I am a rubyist looking for more information on erlang, > and I was > > hoping to find some resources... If you would be so kind as to let > me know > > any that have helped you. > > In addition to the books and the online resources mentioned... > > I find it helps to pick a project that you'd be comfortable solving in > another language, then tackle it in Erlang. Solve mini problems as you > go. Ask questions here. > > I find http://erldocs.com/ helpful as a reference. > > Garrett > > P.S. Come to think of it, I don't know if > http://www.erlang.org/doc.html was mentioned. None of the books are a > substitute for that. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Jul 12 01:39:33 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 11 Jul 2011 16:39:33 -0700 Subject: [erlang-questions] Erlang-Java interface through ordinary ports In-Reply-To: References: Message-ID: <4E1B89B5.2020005@gmail.com> Hi Dan, It is easiest to use the jinterface library for this task, since it is included within OTP/Erlang. The version value is included within the jinterface code that is needed for port communication. Most people use system out/in for ports, but then you are unable to get any print statements, which can be annoying... and often stderr is lost. If you look at CloudI (http://cloudi.org/), it makes sure to open a port (using C++ loader code to exec the JVM) which starts a bunch of sockets with deterministic file descriptor numbers (starting at 3), then uses the jinterface library to encode/decode Erlang data used in the Java CloudI API. So, the example is more complex, but it avoids stdout/stderr (1/2) and handles those separately within the C++ loader code (sending it back as port communication). The higher-level usage of jinterface is similar to what you want, if you ignore some of the magic to make the integer file descriptors work within the java source code (https://github.com/okeuday/CloudI/blob/master/src/api/java/org/cloudi/API.java). - Michael On 07/11/2011 03:14 PM, Daniel Dormont wrote: > Hello Erlangers, > > I would like to have an Erlang application (specifically the auth_external module in Ejabberd) communicate with a Java program I am writing. Not as a node, but rather using the port driver API, since that is what Ejabberd already supports. From the information I've seen, it shouldn't be too terribly hard - just listen for data in System.in, look for a two-byte chunk that when read as a short specifies the number of additional bytes to read, and repeat. I'm just wondering if this is something that's considered normal and reasonable to do, any gotchas I should be aware of, etc. > > The only reason I'm picking Java is the rest of my application is Java and I'd like to reuse some libraries and such. > > thanks, > Dan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Tue Jul 12 01:39:38 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Mon, 11 Jul 2011 18:39:38 -0500 Subject: [erlang-questions] Erlang-Java interface through ordinary ports In-Reply-To: References: Message-ID: Sounds reasonable to me. On Mon, Jul 11, 2011 at 5:14 PM, Daniel Dormont wrote: > Hello Erlangers, > > I would like to have an Erlang application (specifically the auth_external > module in Ejabberd) communicate with a Java program I am writing. Not as a > node, but rather using the port driver API, since that is what Ejabberd > already supports. From the information I've seen, it shouldn't be too > terribly hard - just listen for data in System.in, look for a two-byte chunk > that when read as a short specifies the number of additional bytes to read, > and repeat. I'm just wondering if this is something that's considered normal > and reasonable to do, any gotchas I should be aware of, etc. > > The only reason I'm picking Java is the rest of my application is Java and > I'd like to reuse some libraries and such. > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Jul 12 03:15:35 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 12 Jul 2011 02:15:35 +0100 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: >> For me, the compelling reason to have *something* like this is that >> function composition and partial application are the bread and butter >> of my programming style, > > This is not unlike saying "assignment and goto are the bread and > buffer of my C style, so Erlang should support assignment and goto". > Function composition and partial application are not part of >>>Erlang's<< native style. > > Again, the old joke: > ? ? ? ?Doctor: What's the problem? > ? ? ? ?Patient: It hurts when I do ? > ? ? ? ?Doctor: Then don't do . > > If you try to write Erlang as if it were SML (or CML), > it's obviously going to hurt. ?So the answer is don't do that. > Ok I can see your point, and to be fair I nowadays spend more time in Erlang than OCaml so I've gotten used to not doing to some extent. >> and in Erlang they both prove to be rather, >> what shall we say, wordy? For example, consider this function (a >> hamcrest matcher) which isn't too bad at all, but is indicative of the >> amount of noise we have to deal with: >> >> -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). >> ends_with(X) -> >> ? ?fun(Y) -> string:equal(string:right(Y, length(X)), X) end. > > The amount of "noise" here is precisely three tokens: 'fun' '->' 'end'. > Your suffering cannot be said to be extreme. > It wasn't an horrific example, as I mentioned, but was meant to be indicative. > As an occasional Haskell and SML programmer, I am certainly not > against Curried functions. ?What I *am* against is an approach which > is ugly, confusing, and almost never applicable. > Can't disagree with you there. > If Erlang syntax were generalised to > > ? ? ? ?ends_with(X)(Y) -> string:equal(string:right(Y, length(X)), X). > > we could eliminate those three tokens. > > What's more, we could do that *WITHOUT* any weird restrictions on > ?- whether pattern matching is allowed in the fun > ?- the form of the fun body > ?- where the arguments appear in the fun body > ?- the order in which the arguments appear in the fun body. > and *WITHOUT* > ?- any weird non-intention-revealing tokens, whether '_' or '?' > Yes but you have to admit, it does look a bit odd. Why can't I just use the variable 'Y' and let the compiler figure out that the function signature needs to change? This was kind of my point - although I went around the houses as was duly (and correctly) chastised for doing so - can the compiler do more of the leg work. >> Not personally, I would much sooner have the compiler deal with the >> anonymous function machinery for me: >> >> -spec(ends_with/1 :: (string()) -> fun((string()) -> boolean())). >> ends_with(X) -> >> ? ?string:equal(string:right(?, length(X)), X) end. > > It would be much much better as > > ends_with(Suffix)(String) -> > ? ?string:equal(string:right(String, length(Suffix)), Suffix). > >> discussing the syntax - I could happily go with any marker-character > > My argument is that ANY marker character is wrong because > ?- you can't nest closures that way (does this ? belong to the > ? inner fun or the outer one?) > ?- you can't read the intent of a parameter from a weirdo like > ? '?' the way you can from a name like String > ?- you have all those other weird restrictions I mentioned above I can understand your dislike of using a marker character. > If you want to do Haskell-like things, it doesn't make sense not > to do it in a Haskell-like way, and that is to say that > > ? ? ? ?()()...() -> > > is equivalent to > > ? ? ? ?() -> > ? ? ? ? ? ?fun () -> > ? ? ? ? ? ? ? ?... > ? ? ? ? ? ? ? ?fun () -> > ? ? ? ? ? ? ? ? ? ? > ? ? ? ? ? ? ? ?end > ? ? ? ? ? ? ? ?... > ? ? ? ? ? ?end > > *NO* new tokens! > *NO* weird restrictions! > Good consistency with the way Clean and ML and Haskell do it. > *NO* changes elsewhere in the language. > Great - is it going to get through the eep process? Do we need to vote on it, contribute (e.g., test proposed patches, etc) or whatever? > Let's see what we have to do now: > > c(F1, F2) -> fun (X) -> F2(F1(X)) end. > c(F1, F2, F3) -> fun (X) -> F3(F2(F1(X))) end. > c(F1, F2, F3, F4) -> fun (X) -> F4(F3(F2(F1(X)))) end. > > doodle(X) -> > ? ?F = c(fun do_thing/1, > ? ? ? ? ?fun calculate_next_thing/1, > ? ? ? ? ?fun get_bored_of_typing_result_x, > ? ? ? ? ?fun start_getting_annoyed_with_lazy_programmers/1), > ? ?{ok, F(X)}. > > Nope, not hard. ?Trivial, in fact. Boilerplate. Might as well type it out inline. There have to be neater ways of composing and currying. I like with your (a1)(a2)(a3) in that it at least makes the application (order) visible and obvious. >> >> I can't understand why the parser requires me to type "fun M:F/A" when >> the "/" character clearly indicates that I'm referencing a function in >> a special way. > > The question is, in a language without static types, how do you > tell the difference between an intentional foo/2 meaning a function > and an erroneous foo/2 that should have been Foo/2, with Foo a > variable bound to a number. We're never going to get static types in Erlang, so I think the answer is that you'd end up with a failing unit or integration test case. Not sure how to do a "raised eyebrow" smiley. > > What _I_ don't understand is why fun F/2 is not allowed with F a > variable. > Yeah, most annoying. >> And if the compiler can recognise a reference to a >> function (local or otherwise), then why not have it recognise "+" as >> an operator for chaining function calls? > > Well, I've done enough mathematics to be driven screaming with rage > up the nearest handy wall at any use of "+" for an operation that is > not commutative. ?The operation you want here is often called ";" in > denotational semantics. ?It's (F;G) X = G(F(X)). Sheesh I hadn't thought about commutativity - yes '+' is a bad choice in light of that. We aren't going to get ';' used in this way are we, given it's existing role in the syntax. Perhaps avoiding overloaded operators would be sensible, though I have to admit that Haskell's use of '.' is very clean, though inadmissible for the same reason. > > Again, in general the compiler *CANNOT* recognise a reference to a > function. ?Haskell and SML manage that by means of type checking, > and *they* don't overload function composition with any other > operation. > The compiler knows full well that ()/ evaluates to a fun!? I'm aware that we don't have type checking at compile time. I'm also vaguely aware of how things work in Haskell/SML. What I'm not aware of is what the combination of *ASTERISK AND CAPITAL LETTERS* means. ;-) From pablo.platt@REDACTED Tue Jul 12 03:16:37 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 11 Jul 2011 18:16:37 -0700 (PDT) Subject: [erlang-questions] aes_cbc_256_decrypt ? Message-ID: <1310433397.7027.YahooMailNeo@web112605.mail.gq1.yahoo.com> The docs says that there is? only 128 bit aes cbc support although openSSL has 256bit support. Is it possible somehow use aes 256 cbc? Maybe call openSSL from erlang? Thanks From tristan.sloughter@REDACTED Tue Jul 12 05:22:24 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 11 Jul 2011 22:22:24 -0500 Subject: [erlang-questions] PaaS for Erlang Message-ID: Does anyone know if any PaaS companies plan to offer Erlang support? I know dotCloud listed Erlang at one point as Alpha -- and Heroku uses Erlang for their routing/mesh layer! It makes sense for them not to spend time on Erlang right away since its not exactly popular for web development :). But I would think that with its growing popularity for implementing other pieces of a web applications stack maybe it would happen one day soon. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jul 12 05:57:08 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 12 Jul 2011 15:57:08 +1200 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> Message-ID: <71EF216F-1EF9-4854-8E2A-C3717FD2B805@cs.otago.ac.nz> On 12/07/2011, at 1:15 PM, Tim Watson wrote: > >> As an occasional Haskell and SML programmer, I am certainly not >> against Curried functions. What I *am* against is an approach which >> is ugly, confusing, and almost never applicable. >> > > Can't disagree with you there. > >> If Erlang syntax were generalised to >> >> ends_with(X)(Y) -> string:equal(string:right(Y, length(X)), X). >> >> we could eliminate those three tokens. >> >> What's more, we could do that *WITHOUT* any weird restrictions on >> - whether pattern matching is allowed in the fun >> - the form of the fun body >> - where the arguments appear in the fun body >> - the order in which the arguments appear in the fun body. >> and *WITHOUT* >> - any weird non-intention-revealing tokens, whether '_' or '?' >> > > Yes but you have to admit, it does look a bit odd. Yes, but that is precisely because it looks just like other languages that do Curried functions. ends_with (x) (y) = rtake y (length x) == x is perfectly good Haskell, assuming an rtake function, and fun ends_with (x) (y) = List.drop (y, length y - length x) = x is perfectly good SML, or would be if it didn't crash when y is shorter than x. > Why can't I just > use the variable 'Y' and let the compiler figure out that the function > signature needs to change? Because it is only in toy examples that a compiler *could* figure out *how* the function signature needs to change. >> If you want to do Haskell-like things, it doesn't make sense not >> to do it in a Haskell-like way, and that is to say that >> >> ()()...() -> >> >> is equivalent to >> >> () -> >> fun () -> >> ... >> fun () -> >> >> end >> ... >> end >> >> *NO* new tokens! >> *NO* weird restrictions! >> Good consistency with the way Clean and ML and Haskell do it. >> *NO* changes elsewhere in the language. >> > > Great - is it going to get through the eep process? Do we need to vote > on it, contribute (e.g., test proposed patches, etc) or whatever? I don't have time to write up an EEP today, but I'll see what I can do. The nice thing about it is that it's entirely in the front end; the rest of the compiler doesn't need to know anything about it. > >> Let's see what we have to do now: >> >> c(F1, F2) -> fun (X) -> F2(F1(X)) end. >> c(F1, F2, F3) -> fun (X) -> F3(F2(F1(X))) end. >> c(F1, F2, F3, F4) -> fun (X) -> F4(F3(F2(F1(X)))) end. >> >> doodle(X) -> >> F = c(fun do_thing/1, >> fun calculate_next_thing/1, >> fun get_bored_of_typing_result_x, >> fun start_getting_annoyed_with_lazy_programmers/1), >> {ok, F(X)}. >> >> Nope, not hard. Trivial, in fact. > > Boilerplate. No, library code, done *ONCE*. The only "boilerplate" would be the -import(compose, [c/4]). % plus any others you want line. It would, in fact, have precisely the same status as function composition in Haskell and SML. Library code. >> The question is, in a language without static types, how do you >> tell the difference between an intentional foo/2 meaning a function >> and an erroneous foo/2 that should have been Foo/2, with Foo a >> variable bound to a number. > > We're never going to get static types in Erlang, so I think the answer > is that you'd end up with a failing unit or integration test case. Not > sure how to do a "raised eyebrow" smiley. We *have* optional static types in Erlang, but you're right that they will probably never be mandatory. Testing and static checking are complementary tools. We need both. Thank goodness we *have* both in Erlang. > The compiler knows full well that ()/ evaluates > to a fun!? Only if preceded by 'fun'. From dmitrii@REDACTED Tue Jul 12 07:33:58 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Tue, 12 Jul 2011 08:33:58 +0300 Subject: [erlang-questions] PaaS for Erlang In-Reply-To: References: Message-ID: http://spawngrid.com/ will open its beta in fall and launch by the end of the year. Probaby with custom solutions such as http://promo.spawngrid.com/zotonic-as-a-service/ as well > Does anyone know if any PaaS companies plan to offer Erlang support? I know dotCloud listed Erlang at one point as Alpha -- and Heroku uses Erlang for their routing/mesh layer! > > It makes sense for them not to spend time on Erlang right away since its not exactly popular for web development :). But I would think that with its growing popularity for implementing other pieces of a web applications stack maybe it would happen one day soon. > > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Tue Jul 12 08:51:46 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Tue, 12 Jul 2011 09:51:46 +0300 Subject: [erlang-questions] Erlang-Java interface through ordinary ports In-Reply-To: References: Message-ID: I had built a library to do just this, including monitoring/linking in case either side dies etc. jinterface takes a little bit of work to get around, but it's certainly possible... my takeaway from this is that it's probably simpler overall to use a node, since you'll probably get into managing serialization, queueing and dispatch by yourself. The difference in effort is marginal either way. On Tue, Jul 12, 2011 at 1:14 AM, Daniel Dormont wrote: > Hello Erlangers, > > I would like to have an Erlang application (specifically the auth_external > module in Ejabberd) communicate with a Java program I am writing. Not as a > node, but rather using the port driver API, since that is what Ejabberd > already supports. From the information I've seen, it shouldn't be too > terribly hard - just listen for data in System.in, look for a two-byte chunk > that when read as a short specifies the number of additional bytes to read, > and repeat. I'm just wondering if this is something that's considered normal > and reasonable to do, any gotchas I should be aware of, etc. > > The only reason I'm picking Java is the rest of my application is Java and > I'd like to reuse some libraries and such. > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hsvhome@REDACTED Tue Jul 12 08:43:29 2011 From: hsvhome@REDACTED (Sergey Shilov) Date: Tue, 12 Jul 2011 09:43:29 +0300 Subject: [erlang-questions] aes_cbc_256_decrypt ? In-Reply-To: <1310433397.7027.YahooMailNeo@web112605.mail.gq1.yahoo.com> References: <1310433397.7027.YahooMailNeo@web112605.mail.gq1.yahoo.com> Message-ID: <527DE765-B483-4BE9-9E60-08077C88F7A3@mail.ru> On Jul 12, 2011, at 4:16 AM, Pablo Platt wrote: > The docs says that there is only 128 bit aes cbc support > > although openSSL has 256bit support. > > > Is it possible somehow use aes 256 cbc? Maybe :-) > Maybe call openSSL from erlang? > Maybe this will help (as an option) ? -------------- next part -------------- A non-text attachment was scrubbed... Name: aes.erl Type: application/octet-stream Size: 51196 bytes Desc: not available URL: -------------- next part -------------- From watson.timothy@REDACTED Tue Jul 12 11:35:20 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 12 Jul 2011 10:35:20 +0100 Subject: [erlang-questions] Matthew Sackman's "cut" proposal In-Reply-To: <71EF216F-1EF9-4854-8E2A-C3717FD2B805@cs.otago.ac.nz> References: <47580784-36BB-42C1-B6B5-A040EB2762CC@cs.otago.ac.nz> <71EF216F-1EF9-4854-8E2A-C3717FD2B805@cs.otago.ac.nz> Message-ID: >> Yes but you have to admit, it does look a bit odd. > > Yes, but that is precisely because it looks just like other languages > that do Curried functions. > > ? ? ? ?ends_with (x) (y) = rtake y (length x) == x > > is perfectly good Haskell, assuming an rtake function, and > > ? ? ? ?fun ends_with (x) (y) = > ? ? ? ? ? ? ? ?List.drop (y, length y - length x) = x > > is perfectly good SML, or would be if it didn't crash when y is > shorter than x. > Yes you're right, they both read perfectly well to me. I guess I have become (oddly) accustomed to Erlang not going things this way, which is deeply ironic given my desire for the feature to appear. >> Why can't I just >> use the variable 'Y' and let the compiler figure out that the function >> signature needs to change? > > Because it is only in toy examples that a compiler *could* figure out > *how* the function signature needs to change. > I'm going to have to take your word for it. Without type signatures, I can see why the compiler can't figure out *what* the variable 'Y' ought to represent, but that it appears in the function body, during function application (at the call site) seems a bit of a hint. I suspect the problem here is that you can't tell the difference between this "toy example" and a typo and other class of programmer error. >> >> Great - is it going to get through the eep process? Do we need to vote >> on it, contribute (e.g., test proposed patches, etc) or whatever? > > I don't have time to write up an EEP today, but I'll see what I can do. > The nice thing about it is that it's entirely in the front end; the > rest of the compiler doesn't need to know anything about it. Well I'd be supportive of this move, and would be quite willing to get involved in testing etc. > >> >>> Let's see what we have to do now: >>> >>> c(F1, F2) -> fun (X) -> F2(F1(X)) end. >>> c(F1, F2, F3) -> fun (X) -> F3(F2(F1(X))) end. >>> c(F1, F2, F3, F4) -> fun (X) -> F4(F3(F2(F1(X)))) end. >>> >>> doodle(X) -> >>> ? ?F = c(fun do_thing/1, >>> ? ? ? ? ?fun calculate_next_thing/1, >>> ? ? ? ? ?fun get_bored_of_typing_result_x, >>> ? ? ? ? ?fun start_getting_annoyed_with_lazy_programmers/1), >>> ? ?{ok, F(X)}. >>> >>> Nope, not hard. ?Trivial, in fact. >> >> Boilerplate. > > No, library code, done *ONCE*. > The only "boilerplate" would be the > -import(compose, [c/4]). % plus any others you want > line. > > It would, in fact, have precisely the same status as function > composition in Haskell and SML. ?Library code. > I see what you're saying. True for Haskell/OCaml - I'm thinking of the lift_n variants and so on, so I guess this is fine. But it will be rather important that this library of code supporting the functional bent, makes it's way relatively quickly into the stdlib so that it's consistently applied and properly supported. >>> The question is, in a language without static types, how do you >>> tell the difference between an intentional foo/2 meaning a function >>> and an erroneous foo/2 that should have been Foo/2, with Foo a >>> variable bound to a number. >> >> We're never going to get static types in Erlang, so I think the answer >> is that you'd end up with a failing unit or integration test case. Not >> sure how to do a "raised eyebrow" smiley. > > We *have* optional static types in Erlang, but you're right that they > will probably never be mandatory. > > Testing and static checking are complementary tools. > We need both. ?Thank goodness we *have* both in Erlang. > Indeed. Dialyzer and PropEr/QuickCheck are tools I wouldn't want to do without. >> The compiler knows full well that ()/ evaluates >> to a fun!? > > Only if preceded by 'fun'. > Yes that's annoyingly true. From vasdeveloper@REDACTED Tue Jul 12 12:07:27 2011 From: vasdeveloper@REDACTED (Kannan) Date: Tue, 12 Jul 2011 15:37:27 +0530 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> Message-ID: Thanks Joe for sharing this paragraph. Reading bulk of black and white pages had never been a problem for me, but how to use , ; - -- in a paragraph. With this paragraph, I understood that as well ;-) On Mon, Jul 11, 2011 at 4:24 PM, Joe Armstrong wrote: > On Mon, Jul 11, 2011 at 2:13 AM, Richard O'Keefe > wrote: > > > > On 9/07/2011, at 2:44 AM, Ulf Wiger wrote: > >> Having said this, I invite anyone who goes through that kind of exercise > to share their results. Not only will it help you evaluate the experiment > honestly; it will increase the store of experiments that can be copied and > tailored to the specific challenges of the next project. > > > > I had an unpleasant experimental experience of my own last year. > > Let me first give you the lesson I learned, and then the background. > > > > LESSON: Expect your experiment to surprise you, > > probably by showing the experiment was a waste of time. > > > > Background: I'm sick of arguments about style. To my mind, it is so > > obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I > > cannotUnderstandWhyOtherPeopleDoNotSeeThat. But they don't. My zeroth, > > the New Zealand Anglican Church has even brought out an electronic > > version of the liturgy called WePray, in a desperate attempt to seem > > hip. (Since it is only available for an operating system sold by what > > may be the largest software company to have been convicted to software > > piracy, I wonder what their ethics committee were doing. But I digress.) > > > > So I devised a little language called Chatterton > > (http://www.cs.otago.ac.nz/cosc345/chatterton.pdf), > > which allowed me to mechanically produce several style variants of > > some sample programs and ask some 3rd year software engineering students > > to find some mistakes in them. > > > > What I expected was one of three things: > > - no measurable effect > > - more readable code (i.e., NOT baStudlyCaps) being easier to fix > > - more familiar (i.e., JustLikeXingJava) being easier to fix. > > > > What I *got* was students telling me they couldn't read code on > > paper; they needed syntax-colouring IDEs (the listings all fitted > > on a single sheet of paper and used black-and-white styling) and > > ideally a debugger so they could find mistakes by stepping through > > the code. I also got students telling me that it was horribly > > unreasonable of me to expect them to read a 30-page manual; NOBODY > > could read that much. > > Oh dear - Attention deficiency syndrome - I suspect, though cannot > prove, that is is > a consequence of our brains being constantly bombarded with irrelevant > information. > > The cure is simple: You set reading them off Proust for a few hours a > day with no interruptions. > > > (for example: from > Cities of the Plain > (Sodom et Gomorrhe) > [Vol. 4 of Remembrance of Things Past-- > (? la Recherche du temps perdu)]) > > > "Their honour precarious, their liberty provisional, lasting only > until the discovery of their crime; their position unstable, like that > of the poet who one day was feasted at every table, applauded in every > theatre in London, and on the next was driven from every lodging, > unable to find a pillow upon which to lay his head, turning the mill > like Samson and saying like him: "The two sexes shall die, each in a > place apart!"; excluded even, save on the days of general disaster > when the majority rally round the victim as the Jews rallied round > Dreyfus, from the sympathy--at times from the society--of their > fellows, in whom they inspire only disgust at seeing themselves as > they are, portrayed in a mirror which, ceasing to flatter them, > accentuates every blemish that they have refused to observe in > themselves, and makes them understand that what they have been calling > their love (a thing to which, playing upon the word, they have by > association annexed all that poetry, painting, music, chivalry, > asceticism have contrived to add to love) springs not from an ideal of > beauty which they have chosen but from an incurable malady; like the > Jews again (save some who will associate only with others of their > race and have always on their lips ritual words and consecrated > pleasantries), shunning one another, seeking out those who are most > directly their opposite, who do not desire their company, pardoning > their rebuffs, moved to ecstasy by their condescension; but also > brought into the company of their own kind by the ostracism that > strikes them, the opprobrium under which they have fallen, having > finally been invested, by a persecution similar to that of Israel, > with the physical and moral characteristics of a race, sometimes > beautiful, often hideous, finding (in spite of all the mockery with > which he who, more closely blended with, better assimilated to the > opposing race, is relatively, in appearance, the least inverted, heaps > upon him who has remained more so) a relief in frequenting the society > of their kind, and even some corroboration of their own life, so much > so that, while steadfastly denying that they are a race (the name of > which is the vilest of insults), those who succeed in concealing the > fact that they belong to it they readily unmask, with a view less to > injuring them, though they have no scruple about that, than to > excusing themselves; and, going in search (as a doctor seeks cases of > appendicitis) of cases of inversion in history, taking pleasure in > recalling that Socrates was one of themselves, as the Israelites claim > that Jesus was one of them, without reflecting that there were no > abnormals when homosexuality was the norm, no anti-Christians before > Christ, that the disgrace alone makes the crime because it has allowed > to survive only those who remained obdurate to every warning, to every > example, to every punishment, by virtue of an innate disposition so > peculiar that it is more repugnant to other men (even though it may be > accompanied by exalted moral qualities) than certain other vices which > exclude those qualities, such as theft, cruelty, breach of faith, > vices better understood and so more readily excused by the generality > of men; forming a freemasonry far more extensive, more powerful and > less suspected than that of the Lodges, for it rests upon an identity > of tastes, needs, habits, dangers, apprenticeship, knowledge, traffic, > glossary, and one in which the members themselves, who intend not to > know one another, recognise one another immediately by natural or > conventional, involuntary or deliberate signs which indicate one of > his congeners to the beggar in the street, in the great nobleman whose > carriage door he is shutting, to the father in the suitor for his > daughter's hand, to him who has sought healing, absolution, defence, > in the doctor, the priest, the barrister to whom he has had recourse; > all of them obliged to protect their own secret but having their part > in a secret shared with the others, which the rest of humanity does > not suspect and which means that to them the most wildly improbable > tales of adventure seem true, for in this romantic, anachronistic life > the ambassador is a bosom friend of the felon, the prince, with a > certain independence of action with which his aristocratic breeding > has furnished him, and which the trembling little cit would lack, on > leaving the duchess's party goes off to confer in private with the > hooligan; a reprobate part of the human whole, but an important part, > suspected where it does not exist, flaunting itself, insolent and > unpunished, where its existence is never guessed; numbering its > adherents everywhere, among the people, in the army, in the church, in > the prison, on the throne; living, in short, at least to a great > extent, in a playful and perilous intimacy with the men of the other > race, provoking them, playing with them by speaking of its vice as of > something alien to it; a game that is rendered easy by the blindness > or duplicity of the others, a game that may be kept up for years until > the day of the scandal, on which these lion-tamers are devoured; until > then, obliged to make a secret of their lives, to turn away their eyes > from the things on which they would naturally fasten them, to fasten > them upon those from which they would naturally turn away, to change > the gender of many of the words in their vocabulary, a social > constraint, slight in comparison with the inward constraint which > their vice, or what is improperly so called, imposes upon them with > regard not so much now to others as to themselves, and in such a way > that to themselves it does not appear a vice." > > One they have got used to reading sentences like this that ramble on > for a few pages without getting anywhere > in particular they will find a 30-page manual easy going. > > /Joe > > > > And finding a definition of an identifier > > in a 2-page listing is just beyond human capacity; it's impossible > > to do that without the machine assistance of an IDE. > > > > So the whole experiment produced no worthwhile data for reasons having > > nothing to do with what I thought I was testing. > > > > As other people have been saying, the "Erlang web servers challenge" > > is at serious risk of producing no worthwhile data. > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greim@REDACTED Tue Jul 12 13:24:55 2011 From: greim@REDACTED (greim) Date: Tue, 12 Jul 2011 13:24:55 +0200 Subject: [erlang-questions] Erlang web servers challenge In-Reply-To: <1D242B7D-C86F-4640-9975-E81305331058@erlang-solutions.com> References: <9148F91E-C865-48BA-BB05-86578E0834D2@pheonic.com> <4F7478D3-7CA5-42C6-9667-04ADDDCDD135@cs.otago.ac.nz> <1D242B7D-C86F-4640-9975-E81305331058@erlang-solutions.com> Message-ID: <4E1C2F07.6060208@schleibinger.com> Hi, i don't like to defend the students here in generally, but specially for the students, who are not studiing Computer Science, but for exampel mechanical or electrical engineering the nowadays hype languages like C++ or Java are much to complicated. Unfortunately most of the code of the (mostly embedded) world, is produced by such guys (like me ;-) ) I have attended a small conference with Niklaus Wirth some weeks ago in Z?rich (videos at: http://www.multimedia.ethz.ch/conferences/2011/oberon ) . Benath some things about Pascal and Oberon the essential resume of the congress was indeed that the hurdle you have to take for these modern ( ?) languages is much to high, and all this not really needed complexity make software unstable. Actual options: 1: lets do everything in Javascript 2: only a small group, very high educated and payed(?) group will write programs in the future Options for a maybe better future: 3: separating the user interface from the core task of your program, but i have seen no really working system doing this. In our company we are making web interfaces, but CGI and HTML is not very smooth and with Ajax we are back to the most complicated solution for doing simple things. 90% of the complexity needed to handle the (graphical) user interfaces. I personally see (core) Erlang more on the Pascal side, but if you ever learned a procedural language, changing to the functional ideas is quite hard. just my five cents Markus From avinash@REDACTED Tue Jul 12 15:05:50 2011 From: avinash@REDACTED (Avinash Dhumane) Date: Tue, 12 Jul 2011 18:35:50 +0530 (IST) Subject: [erlang-questions] Ways to get started In-Reply-To: Message-ID: <1310475950.29307.YahooMailClassic@web5601.biz.mail.in.yahoo.com> How would a Sales person get himself started? No, I am not a programmer. But, I am tasked to sell an abstract form that embodies Erlang/OTP as its DNA. And, my prospects are the CXOs in the enterprise application software space. Please bear with me a little bit so that I can articulate my difficulties. ? I left IONA Technologies in 2002 (I was a programmer at IONA, and worked on point/patch releases of Orbix). It seems CXOs at IONA did not "listen" to Steve's appeal "to use Erlang". So, I take a hint from him that CXOs do not like a proposal or invitation "to use" yet another ?name? of technology even though it can provide them better trade-offs than what they have chosen for now and which are burning their pockets badly. And, I have testified this at sufficient number of places in the field. So, I am starting with this one-liner pitch sans the word ?Erlang? to draw the attention of prospects: "Fulfil SLAs economically in the presence of faults and extreme load." (Yes, I picked up this from Joe?s thesis, and twisted it for my sales purpose) ? Obviously, I need to refine this one-liner further as I go down the hierarchy of ?Buy? decision makers in my sales drill at the prospect's shop. I need to show them how their (business) application architectures have developed the *resistance* property that simply resists reliability, scalability and extensibility, which in turn costs them money. And, by architecture, I mean the things mentioned here: http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm. ?The abstract form that I propose to sell, then, begins to take concrete form in the context of the (business) function ? form follows function ? which the prospect sees as bottleneck or money-leaking hole. I seize it as an opportunity to insert my form (made of Erlang as one of the central architectural elements) as a gold-link in the midst of several weak-links that the prospect has. The goal really is to achieve 20::80 (effort::mileage) ratio for the prospect and a ?beach head? for me (for subsequent invasion in the customer shop). ? I end up presenting my abstract form sometimes as an application container, sometimes as ESB, sometimes as DBMS, sometimes as TPM (the programmers in my lab implement Paxos protocols), sometimes as XTP; I basically measure the water-depth of the prospect and his drifts and the technology buzzwords he subscribes to and try to take an ?interesting? standpoint for him so that I can resonate with him and get him to speak about his pain points! I have deliberately kept my form as abstract as possible, because Erlang lets me concretize it rapidly with its economy of expression for a specific need of the prospect to overcome the resistance he faces in the application architectures he has put together, often, forcibly! ? Well, my difficulty is that I was an average programmer in imperative programming language a decade ago and since then had not been up to speed with the ?thoughts? (as oppose to ?details?) behind mainstream technologies out there in the field bleeding the enterprises. This might be a blessing in disguise since I do not have to unlearn too many things for I haven?t gone down the many roads that lead to failures, but I still need to be equipped with the architectural elements that build the rustiness (resistance) in the systems and be able to convey those to the potential buyers to help them understand why they are bleeding and what I have got to offer them. However, I am not able to pick up the branches of evolution of the technologies, along with their characteristics, that have made it to the field and unfortunately commanding the sizeable market share. The data that I have to scan is way too voluminous, and I don?t know where to get started and proceed thereafter. ? Any insights or directions? ? Many thanks! ? --- On Tue, 12/7/11, Joe Armstrong wrote: From: Joe Armstrong Subject: Re: [erlang-questions] Ways to get started To: "Mihai Balea" Cc: erlang-questions@REDACTED Date: Tuesday, 12 July, 2011, 2:19 AM After 30 years you will get the hang of this and be a good programmer. Tools are no substitute for typing in small programs and understanding exactly how they work. This is true for all programming language. Programming is an art form, there is no easy way. Like playing the violin - is there an easy way of playing the violin other than by practising for thousands of hours? I think not. /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Tue Jul 12 15:59:42 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 12 Jul 2011 08:59:42 -0500 Subject: [erlang-questions] PaaS for Erlang In-Reply-To: References: Message-ID: Awesome! I signed up for the beta. On Tue, Jul 12, 2011 at 12:33 AM, Dmitrii Dimandt wrote: > > http://spawngrid.com/ will open its beta in fall and launch by the end of > the year. Probaby with custom solutions such as > http://promo.spawngrid.com/zotonic-as-a-service/ as well > > Does anyone know if any PaaS companies plan to offer Erlang support? I know > dotCloud listed Erlang at one point as Alpha -- and Heroku uses Erlang for > their routing/mesh layer! > > It makes sense for them not to spend time on Erlang right away since its > not exactly popular for web development :). But I would think that with its > growing popularity for implementing other pieces of a web applications stack > maybe it would happen one day soon. > > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > =================================== > Dmitrii Dimandt > dmitrii@REDACTED > > ------------------------------------------------------------ > Erlang in Russian > http://erlanger.ru/ > > TurkeyTPS > http://turkeytps.com/ > ------------------------------------------------------------ > > LinkedIn: http://www.linkedin.com/in/dmitriid > GitHub: https://github.com/dmitriid > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Jul 12 16:50:17 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 12 Jul 2011 16:50:17 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: <1310475950.29307.YahooMailClassic@web5601.biz.mail.in.yahoo.com> References: <1310475950.29307.YahooMailClassic@web5601.biz.mail.in.yahoo.com> Message-ID: On Tue, Jul 12, 2011 at 3:05 PM, Avinash Dhumane wrote: > How would a Sales person get himself started? No, I am not a programmer. > But, I am tasked to sell an abstract form that embodies Erlang/OTP as its > DNA. And, my prospects are the CXOs in the enterprise application software > space. Please bear with me a little bit so that I can articulate my > difficulties. > Tricky - when I had company we always sent two people to a sales meeting. The sales guy + a programmer. The sales guy does the "pitch" and the programmer answers the technical questions. It's not good to send a sales guy into a group of programmers - they get found-out after a few minutes. It's not good to send a programmer to a meeting. programmers don't bother with follow-up calls and "what do we do next" type stuff - on a good day they will try to solve all the customers problems in the sales meeting and be done with it. One of the great success factors of Cisco was they always sent engineers to sales meetings. > ** ** > > I left IONA Technologies in 2002 (I was a programmer at IONA, and worked on > point/patch releases of Orbix). It seems CXOs at IONA did not "listen" to > Steve's appeal "to use Erlang". So, I take a hint from him that CXOs do not > like a proposal or invitation "to use" yet another ?name? of technology even > though it can provide them better trade-offs than what they have chosen for > now and which are burning their pockets badly. And, I have testified this at > sufficient number of places in the field. So, I am starting with this > one-liner pitch sans the word ?Erlang? to draw the attention of prospects: > "Fulfil SLAs economically in the presence of faults and extreme load." (Yes, > I picked up this from Joe?s thesis, and twisted it for my sales purpose) > > ** ** > > Obviously, I need to refine this one-liner further as I go down the > hierarchy of ?Buy? decision makers in my sales drill at the prospect's shop. > I need to show them how their (business) application architectures have > developed the *resistance* property that simply resists reliability, > scalability and extensibility, which in turn costs them money. And, by > architecture, I mean the things mentioned here: > http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm. The abstract > form that I propose to sell, then, begins to take concrete form in the > context of the (business) function ? form follows function ? which the > prospect sees as bottleneck or money-leaking hole. I seize it as an > opportunity to insert my form (made of Erlang as one of the central > architectural elements) as a gold-link in the midst of several weak-links > that the prospect has. The goal really is to achieve 20::80 > (effort::mileage) ratio for the prospect and a ?beach head? for me (for > subsequent invasion in the customer shop). > > ** ** > > I end up presenting my abstract form sometimes as an application container, > sometimes as ESB, sometimes as DBMS, sometimes as TPM (the programmers in my > lab implement Paxos protocols), sometimes as XTP; I basically measure the > water-depth of the prospect and his drifts and the technology buzzwords he > subscribes to and try to take an ?interesting? standpoint for him so that I > can resonate with him and get him to speak about his pain points! I have > deliberately kept my form as abstract as possible, because Erlang lets me > concretize it rapidly with its economy of expression for a specific need of > the prospect to overcome the resistance he faces in the application > architectures he has put together, often, forcibly! > If I were in the room when you did this I would ask what all the abbreviations stood for and ask how Paxos worked. I have a vague idea how paxos works and would be keen to learn more. This is why you need your technical guy in the sales group. > ** ** > > Well, my difficulty is that I was an average programmer in imperative > programming language a decade ago and since then had not been up to speed > with the ?thoughts? (as oppose to ?details?) behind mainstream technologies > out there in the field bleeding the enterprises. This might be a blessing in > disguise since I do not have to unlearn too many things for I haven?t gone > down the many roads that lead to failures, but I still need to be equipped > with the architectural elements that build the rustiness (resistance) in the > systems and be able to convey those to the potential buyers to help them > understand why they are bleeding and what I have got to offer them. However, > I am not able to pick up the branches of evolution of the technologies, > along with their characteristics, that have made it to the field and > unfortunately commanding the sizeable market share. The data that I have to > scan is way too voluminous, and I don?t know where to get started and > proceed thereafter. > > ** ** > > Any insights or directions? > No - it's difficult - if sales were easy and you could just follow some magic formula then I guess all salesmen would be stinking rich - since they aren't then it must be tricky. Jane Walerud a blutail co-founder said that "it was all about the relationship" - feelings warm fuzzy stuff that programmers don't understand - she was probably right. Most of the programmers I know are the worse salesmen on the planet - they don't want to buy software because they'd rather write it themselves and mostly they want to give away what they have done not sell it. Cheers - and sorry I can't help more /Joe ** ** > > Many thanks! > > ** ** > > --- On *Tue, 12/7/11, Joe Armstrong * wrote: > > > From: Joe Armstrong > > Subject: Re: [erlang-questions] Ways to get started > To: "Mihai Balea" > > Cc: erlang-questions@REDACTED > Date: Tuesday, 12 July, 2011, 2:19 AM > > > After 30 years you will get the hang of this and be a good programmer. > > Tools are no substitute for typing in small programs and > understanding exactly how they work. This is true for all programming > language. Programming is an art form, there is no easy way. > Like playing the violin - is there an easy way of playing the violin other > than by practising for thousands of hours? I think not. > > /Joe > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jul 12 16:58:00 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 12 Jul 2011 16:58:00 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Mon, Jul 11, 2011 at 22:49, Joe Armstrong wrote: > > Your brain is a zillion times better than the best IDE. programs > form in you brain not in an IDE. > Very true. I often sit pondering on a problem. Ride the bike and ponder on the problem. Play computer games while the brain ponders on the problem unconsciously. Go see Rich Hickey's "Hammock Driven Development", http://blip.tv/clojure/hammock-driven-development-4475586 Then, as soon as the solution to the problem forms in the brain, it is often a quick task of 10-15 minutes to write the 50-100 lines of code which solves it. The only thing that makes the old guys seem to darn effective at producing code is that they cheat. They have done it before and they just have to alter small things to make it right. And they know what code not to write (Code lines are spent, not produced - http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1036.html ). The hard part of programming is to get the right solution. The tools you use matter almost nothing as they usually don't help you get the right solution. The point in addition is focus. If you have focus on getting a git repo on github, focus on configuring and installing rebar, focus on getting an IDE - then you are not spending time on understanding Erlang. A virtue I learned as a programmer is to keep focus on one thing at a time. Whenever you are on a task and another thing you should do pops up, write it down somewhere and ignore it for now. Our brains, when we work, are not multi-core machines. You can always go back and add a build system when the code you are writing gets too big to handle as separate compiles. You can always add Version Control later. Programs are like seeds. They are planted and then they grow under nurture and care. Don't plan for the Giant Sequoia when all you have is a sapling. -- J. From g@REDACTED Tue Jul 12 17:37:00 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 12 Jul 2011 09:37:00 -0600 Subject: [erlang-questions] Ways to get started In-Reply-To: <1310475950.29307.YahooMailClassic@web5601.biz.mail.in.yahoo.com> References: <1310475950.29307.YahooMailClassic@web5601.biz.mail.in.yahoo.com> Message-ID: On Tue, Jul 12, 2011 at 7:05 AM, Avinash Dhumane wrote: > > How would a Sales person get himself started? No, I am not a programmer. But, I > am tasked to sell an abstract form that embodies Erlang/OTP as its DNA. And, my > prospects are the CXOs in the enterprise application software space. Please bear > with me a little bit so that I can articulate my difficulties. I've sold software and developer services and I'm also a programmer. IMO selling packaged software is a completely different ball of wax from services. I'm not sure what an "abstract form" is. If what you have is a product, I think leading with the underlying language is a mistake. Surely there's some business value to focus on. Regarding Erlang/OTP you might just keep it high level: "You guys heard of five nines? How about nine nines!" and reference the host of Erlang success stories. That's a truth stretching exercise that people on this list will wince at, but since you're the "sales guy" you get a bit of a pass. If you get drilled on the details, as Joe mentioned, punt over to your sales support engineer. If she can't be there in person, have her on speaker phone. If you're selling services, I think you're in difficult waters. Unless you have a deep track record in some vertical or some leveragable asset, pitching an architecture, at least in my experience, will be all up hill. If you have that track record or asset, lead with its customer value. The technical underpinnings are generally slated for side discussions. Best of luck -- I'd love to see Erlang make better in roads in the CXO circles! Garrett From carlo.bertoldi@REDACTED Tue Jul 12 18:06:30 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Tue, 12 Jul 2011 18:06:30 +0200 Subject: [erlang-questions] remsh Message-ID: <4E1C7106.6080105@ubiquity.it> Hello list, I'm trying to connect to a remote node and execute a function. If I do it the manual way, it works: erl -sname foo -remsh prova@REDACTED 1> probe_db_manager:cleanup(). true But if I try to do it entirely from the cli it gets angry: erl -sname foo -remsh prova@REDACTED -run probe_db_manager cleanup {"init terminating in do_boot",{noproc,{gen_server,call,[{global,probe_db_manager},cleanup]}}} probe_db_manager is a gen_server process. I also tried this: erl -sname foo -remsh prova@REDACTED -run global whereis_name probe_db_manager and it returns nothing. Thanks, Carlo From essen@REDACTED Tue Jul 12 18:27:21 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 12 Jul 2011 18:27:21 +0200 Subject: [erlang-questions] inet.erl socket_setopt type incomplete In-Reply-To: <4E15FE64.9010709@2600hz.com> References: <4E15FE64.9010709@2600hz.com> Message-ID: <4E1C75E9.4050000@dev-extend.eu> This probably should be in erlang-bugs@ instead. I confirm having seen this issue before too. On 07/07/2011 08:43 PM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > In R14B03, the type socket_setopt() in inet.erl does not allow for the > convenience options 'binary' and 'list' (which internally get translated > to {'mode', 'binary' | 'list'} which *is* in the type), and this causes > Dialyzer to complain about inet:setopts(Socket, [{active, once}, > binary]) (in my case). > > Setting the options list to [{active, once}, {mode, binary}] alleviates > the Dialyzer warning but doesn't reflect what the function actually allows. > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJOFf5kAAoJENc77s1OYoGgCAsH/3avCJlBdnJ0c78naoclMWre > +AC9amjmTrVCkp6dg5mkCnMFsGpImCba171qMx3005pVS/e123px+6ek86cda9Gy > rhIusElV8TR6r08QNzcbcdEHW89vbzYN6Gmc0AhnJX1gS6yBQ6vhn9tH3HxgaroZ > N3VZCUAuyKaun2sWii7WfI0LdtxmWBxsZmUnpywPZhNOFm9A0hyY1rH533y+KIKJ > kfleH3iO2E4tasE9U8eRNX2E9+fM00sXlVE84RUxVAonMcRTZyqBFxBa3nRStgii > ShClwNH3CXnRiCgUgZhu6bW7Zd1Umq7Ru3gRRxcJTrTMJ2T8MfsD8EQ8/rgtVhw= > =qgkH > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Dev:Extend From erlang@REDACTED Tue Jul 12 22:01:35 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 12 Jul 2011 22:01:35 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: On Tue, Jul 12, 2011 at 4:58 PM, Jesper Louis Andersen wrote: > On Mon, Jul 11, 2011 at 22:49, Joe Armstrong wrote: >> >> Your brain is a zillion times better than the best IDE. programs >> form in you brain not in an IDE. >> > > Very true. I often sit pondering on a problem. Ride the bike and > ponder on the problem. Play computer games while the brain ponders on > the problem unconsciously. Go see Rich Hickey's "Hammock Driven > Development", > > http://blip.tv/clojure/hammock-driven-development-4475586 > > Then, as soon as the solution to the problem forms in the brain, it is > often a quick task of 10-15 minutes to write the 50-100 lines of code > which solves it. The only thing that makes the old guys seem to darn > effective at producing code is that they cheat. They have done it > before and they just have to alter small things to make it right. And > they know what code not to write (Code lines are spent, not produced - > http://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/EWD1036.html > ). > > The hard part of programming is to get the right solution. The tools > you use matter almost nothing as they usually don't help you get the > right solution. Yes - at a rough guess 90% of time = understanding the problem 10% coding the final version (in the 90% I'll write lots of code to throw away - no point in having a revision control system for this bit) - I don't want to keep it As for the gits etc. I just learn half a dozen commands and stick to them Time spent of fixing your tools is time not spent on thinking about the problem. The most efficient code is the code that is never written - I throw away (say) 50% of all the code I write. When I was a beginner I would throw away 75% my goal would be to throw away 0% that would mean I had though long and hard enough before writing the code to only write good that was good enough to keep. (These are only rough figures - I guess I reckon on writing things two or three times, years ago was up at six times, Knuth says everything should be rewritten seven times) > > The point in addition is focus. If you have focus on getting a git > repo on github, focus on configuring and installing rebar, focus on > getting an IDE - then you are not spending time on understanding > Erlang. Yup - also I cannot really understand why people want to learn specialist tools for the development of things in language X if such tools cannot be used with language Y. rebar is great if you understand what it is abstracting, otherwise useless. I use emacs, shell scripts, make and xterm for *everything*. I use this for writing erlang,c,java,javascript,TCL,assembler,bash scripts, makefiles, html, css, I also make sure that emacs, make etc. work the same way on OS-X,ubuntu and windows. In the early 80's I wrote about 150K lines of FORTRAN with a line editor (full screen editors had not been invented) all the files were in one directory (the OS didn't have hierarchal directories) and there was a ten character limit on file names. There was no revision control system - 10 programmers worked on the same system. I don't actually believe that modern tools would have made me significantly more efficient. To circumvent the limitations we invented ad hock versions of email, revision control, etc. Not fancy things but they got the job done. As today most of my time in the 80's was spent understanding the problem - since this takes 90% of the time providing tool support in the other 10% is not likely to help productivty. Programming languages hit directly at the 90% because they are tools for thought - GIT etc does not help me think. Prolog/Haskell/Erlang provide ways of thought that are superior to C/C++/Java. Give me better ways of thinking with bad tools support rather than bad ways of thinking with good tool support. I still use printf debugging in all languages. I think that things like debugging cause compensation mechanisms to come into place - if language X is very difficult to debug then you think a little harder before writing the program. Also true of fancy revision control systems - if I've got one fine, If not I'll compensate by thinking a little more. Most programming takes place in my head - tapping the stuff in is the last part of a long and mostly invisible process. There should be a name for this development process. Zen development - no tools - just your brain - all we are doing in making pleasing patterns of zeros and ones that follow the way of programming. If we follow the way the programs will please us. /Joe > A virtue I learned as a programmer is to keep focus on one > thing at a time. Whenever you are on a task and another thing you > should do pops up, write it down somewhere and ignore it for now. Our > brains, when we work, are not multi-core machines. > > You can always go back and add a build system when the code you are > writing gets too big to handle as separate compiles. You can always > add Version Control later. Programs are like seeds. They are planted > and then they grow under nurture and care. Don't plan for the Giant > Sequoia when all you have is a sapling. > > -- > J. > From matthew.hillsborough@REDACTED Tue Jul 12 22:25:48 2011 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Tue, 12 Jul 2011 16:25:48 -0400 Subject: [erlang-questions] Is Erlang a good tool for this particular project? Message-ID: Greetings Erlang community, Let me further elaborate on my question that's in the subject of this message. I tried to reach out with this question on StackOverflow, however I did not have much luck there. Perhaps the community here can provide some feedback here for me to let me know if I'm on the right track or if Erlang is not the right tool for what I'm trying to accomplish. I'm building native mobile applications in both iOS and Android. These apps require "realtime" updates from and to the server, same as many (but not all) network-based application does (Facebook, social games like Words with Friends, Finance applications, etc). The communication here is bi-directional, in the sense that the server might have updates for the mobile clients and the clients will be pushing data down to the server whenever necessary. I think using HTTP long polling for this is over kill in the sense that long polling can be detrimental to battery life, especially with a lot of TCP setup/teardown for every HTTP connection the device needs to send out through the wire. It might make sense to have the mobile applications use persistent TCP sockets to establish a connection to the server, and send RPC style commands to the server for all web service communication. This ofcourse, would require a server to handle the long-lived TCP connection and be able to speak to a web service once it makes sense of the data passed down the TCP pipe. I'm thinking of passing data in plain text using JSON or XML and then using some kind of Erlang interface to HTTP to call a web service to handle all the REST type communication. The responses would then go back to the "RPC" Erlang instance, which would send the updates to the appropriate client(s). Perhaps an Erlang based RPC server would do well for a network based application like this. It would allow for the mobile apps to send and receive data from the server all over one connection without multiple setup/tear down that individual HTTP requests would do. Since no web browser is involved, we do not need to deal with the nuances of HTTP long-polling at the mobile client level. I also haven't seen great long polling/keep-alive support on the client-side in iOS, but that's irrelevant for the community here. A lot of these "COMET" and long-polling/streaming servers are built with HTTP in mind. I'm thinking just using a plain-text protocol over TCP is better catered for the type of app I'm building, will make the client more responsive, allow for receiving of updates from the server without constantly polling the server, etc. I also looked into HTTP pipelining, but it doesn't look to be worth the trouble when it comes to implementing it on the clients. Also, I'm not sure if it would allow for bi-directional communication in the client<->server communication channel. Am I completely out of line in thinking that building a custom solution in Erlang is a good idea here? To my understanding, Erlang excels at servers like this, and if I run the server on tcp/80, I should be able to avoid most firewall/port issues. The client would need work to deal with timeouts, re connections, acknowledging receipt of asynchronous requests, but that's not Erlang's problem. Has anyone built something similar before? Should I just stick to a web server and deal with "COMET" type technologies? (WebSockets, long-polling, client-side polling). Was hoping someone could solidify that I'm not entirely insane for wanting a better solution than HTTP would serve in this case, at least at the client level. I'll still be using HTTP/REST extensively, the Erlang server would just handle the persistent connections and messaging to the Web Service (which would probably be something like Django or Rails). Sorry for the long post; I am just excited to get into the heads of people who are smarter than I. Happy hacking! Matthew -------------- next part -------------- An HTML attachment was scrubbed... URL: From peralta.alejandro@REDACTED Tue Jul 12 23:03:26 2011 From: peralta.alejandro@REDACTED (Ale) Date: Tue, 12 Jul 2011 18:03:26 -0300 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: Hi, Just a thought here... Erlang is great for scalability and handling multiple connections, and well I'm sure a lot of people can give you a better description fo this, but it occurred to me if you considered other protocols instead of HTTP? The problem seems to be there rather than the language you write your sever. It occurred to me that you might benefit from using Jabber/XMPP. Googling I found some iOS jabber clients http://code.google.com/p/xmppframework/ and the cannonical jabber server is written in Erlang. Regards, 2011/7/12 Matthew Hillsborough : > Greetings Erlang community, > Let me further elaborate on my question that's in the subject of this > message. I tried to reach out with this question on StackOverflow, however I > did not have much luck there. Perhaps the community here can provide some > feedback here for me to let me know if I'm on the right track or if Erlang > is not the right tool for what I'm trying to accomplish. > > I'm building native mobile applications in both iOS and Android. These apps > require "realtime" updates from and to the server, same as many (but not > all) network-based application does (Facebook, social games like Words with > Friends, Finance applications, etc). The communication here is > bi-directional, in the sense that the server might have updates for the > mobile clients and the clients will be pushing data down to the server > whenever necessary. > > I think using HTTP long polling for this is over kill in the sense that long > polling can be detrimental to battery life, especially with a lot of TCP > setup/teardown for every HTTP connection the device needs to send out > through the wire. It might make sense to have the mobile applications use > persistent TCP sockets to establish a connection to the server, and send RPC > style commands to the server for all web service communication. This > ofcourse, would require a server to handle the long-lived TCP connection and > be able to speak to a web service once it makes sense of the data passed > down the TCP pipe. I'm thinking of passing data in plain text using JSON or > XML and then using some kind of Erlang interface to HTTP to call a web > service to handle all the REST type communication. The responses would then > go back to the "RPC" Erlang instance, which would send the updates to the > appropriate client(s). > > Perhaps an Erlang based RPC server would do well for a network based > application like this. It would allow for the mobile apps to send and > receive data from the server all over one connection without multiple > setup/tear down that individual HTTP requests would do. Since no web browser > is involved, we do not need to deal with the nuances of HTTP long-polling at > the mobile client level. I also haven't seen great long polling/keep-alive > support on the client-side in iOS, but that's irrelevant for the community > here. > > A lot of these "COMET" and long-polling/streaming servers are built with > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > better catered for the type of app I'm building, will make the client more > responsive, allow for receiving of updates from the server without > constantly polling the server, etc. > > I also looked into HTTP pipelining, but it doesn't look to be worth the > trouble when it comes to implementing it on the clients. Also, I'm not sure > if it would allow for bi-directional communication in the client<->server > communication channel. > > Am I completely out of line in thinking that building a custom solution in > Erlang is a good idea here? To my understanding, Erlang excels at servers > like this, and if I run the server on tcp/80, I should be able to avoid most > firewall/port issues. The client would need work to deal with timeouts, re > connections, acknowledging receipt of asynchronous requests, but that's not > Erlang's problem. > > Has anyone built something similar before? Should I just stick to a web > server and deal with "COMET" type technologies? (WebSockets, long-polling, > client-side polling). > > Was hoping someone could solidify that I'm not entirely insane for wanting a > better solution than HTTP would serve in this case, at least at the client > level. I'll still be using HTTP/REST extensively, the Erlang server would > just handle the persistent connections and messaging to the Web Service > (which would probably be something like Django or Rails). > > Sorry for the long post; I am just excited to get into the heads of people > who are smarter than I. > > Happy hacking! > > Matthew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Ale. From daniel.goertzen@REDACTED Tue Jul 12 23:25:37 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 12 Jul 2011 16:25:37 -0500 Subject: [erlang-questions] edoc: documenting "-type" Message-ID: I am trying to document a "-type" close to the top of my modules like this... %% @doc This type is awesome. Really, it is great. -type key() :: binary() | atom(). ... but it collides with my module @doc entry above it ("multiple @doc tag" error). Is there a right way to document types or is this not supported right now? The generated documentation shows all my types nicely, but I'd love to have a bit of explanation appear with them. Thanks, Dan. From therevoltingx@REDACTED Tue Jul 12 23:32:44 2011 From: therevoltingx@REDACTED (Miguel Morales) Date: Tue, 12 Jul 2011 14:32:44 -0700 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: I agree with the above poster, sounds like something like XMPP would be good for. There is a smack library for Android that I've heard good things about. When I was starting, because there was nothing specific to mobile devices I ended up using RabbitMQ and its Java bindings to build a realtime service that users could push/subscribe to. The way I do it, is on the server side I have some queue setup and a bunch of worker processes running. When the server gets a new message from the clients, one of the workers processes it and updates the client information. If the workers are overloaded, the message is left in the queue until it can be processed. Polling on a device will shorten the battery life. From some tests I've ran (and I suspect is how google implements its C2DM service) it appears that having a socket open doesn't activate the radio, it's only activated until data is ready to be sent or read. You can also implement RPC with a MQ server. Although, it's not strictly bi-directional. On Tue, Jul 12, 2011 at 1:25 PM, Matthew Hillsborough wrote: > Greetings Erlang community, > Let me further elaborate on my question that's in the subject of this > message. I tried to reach out with this question on StackOverflow, however I > did not have much luck there. Perhaps the community here can provide some > feedback here for me to let me know if I'm on the right track or if Erlang > is not the right tool for what I'm trying to accomplish. > > I'm building native mobile applications in both iOS and Android. These apps > require "realtime" updates from and to the server, same as many (but not > all) network-based application does (Facebook, social games like Words with > Friends, Finance applications, etc). The communication here is > bi-directional, in the sense that the server might have updates for the > mobile clients and the clients will be pushing data down to the server > whenever necessary. > > I think using HTTP long polling for this is over kill in the sense that long > polling can be detrimental to battery life, especially with a lot of TCP > setup/teardown for every HTTP connection the device needs to send out > through the wire. It might make sense to have the mobile applications use > persistent TCP sockets to establish a connection to the server, and send RPC > style commands to the server for all web service communication. This > ofcourse, would require a server to handle the long-lived TCP connection and > be able to speak to a web service once it makes sense of the data passed > down the TCP pipe. I'm thinking of passing data in plain text using JSON or > XML and then using some kind of Erlang interface to HTTP to call a web > service to handle all the REST type communication. The responses would then > go back to the "RPC" Erlang instance, which would send the updates to the > appropriate client(s). > > Perhaps an Erlang based RPC server would do well for a network based > application like this. It would allow for the mobile apps to send and > receive data from the server all over one connection without multiple > setup/tear down that individual HTTP requests would do. Since no web browser > is involved, we do not need to deal with the nuances of HTTP long-polling at > the mobile client level. I also haven't seen great long polling/keep-alive > support on the client-side in iOS, but that's irrelevant for the community > here. > > A lot of these "COMET" and long-polling/streaming servers are built with > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > better catered for the type of app I'm building, will make the client more > responsive, allow for receiving of updates from the server without > constantly polling the server, etc. > > I also looked into HTTP pipelining, but it doesn't look to be worth the > trouble when it comes to implementing it on the clients. Also, I'm not sure > if it would allow for bi-directional communication in the client<->server > communication channel. > > Am I completely out of line in thinking that building a custom solution in > Erlang is a good idea here? To my understanding, Erlang excels at servers > like this, and if I run the server on tcp/80, I should be able to avoid most > firewall/port issues. The client would need work to deal with timeouts, re > connections, acknowledging receipt of asynchronous requests, but that's not > Erlang's problem. > > Has anyone built something similar before? Should I just stick to a web > server and deal with "COMET" type technologies? (WebSockets, long-polling, > client-side polling). > > Was hoping someone could solidify that I'm not entirely insane for wanting a > better solution than HTTP would serve in this case, at least at the client > level. I'll still be using HTTP/REST extensively, the Erlang server would > just handle the persistent connections and messaging to the Web Service > (which would probably be something like Django or Rails). > > Sorry for the long post; I am just excited to get into the heads of people > who are smarter than I. > > Happy hacking! > > Matthew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- ~ Jeremiah:9:23-24 Android 2D MMORPG: http://solrpg.com/,?http://www.youtube.com/user/revoltingx From tristan.sloughter@REDACTED Wed Jul 13 01:26:19 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 12 Jul 2011 18:26:19 -0500 Subject: [erlang-questions] Checking Types at Runtime Message-ID: I'm pretty sure this isn't possible but I wanted to be sure before working around the lack of it. if I have a type and record like: -type password() :: binary(). -record(user, {username :: binary(), password :: password()}. At run time there is no way to tell a field is suppose to be of some type during runtime, right? Basically I'd like to be able to have generic or generated create functions for records that may have to modify arguments (like a password has to be encrypted from the string provided) when creating the record. So the create function would pass each field, its value and its type to a transform function that matches on type and returns the appropriately modified value. Any ideas on how to do this with type definitions or do I need to use tuples and an atom like {Type, Value} for the value of every field to achieve this? Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jul 13 01:52:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 13 Jul 2011 00:52:13 +0100 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: On 12 July 2011 22:32, Miguel Morales wrote: > I agree with the above poster, sounds like something like XMPP would > be good for. ?There is a smack library for Android that I've heard > good things about. ?When I was starting, because there was nothing > specific to mobile devices I ended up using RabbitMQ and its Java > bindings to build a realtime service that users could push/subscribe > to. Also, if your target environment (e.g., IOS) makes HTTP necessary, due to protocol restrictions or whatever, most of these messaging and/or integration technologies tend to support some kind of transport over HTTP anyway. So for example, you can do XMPP over an HTTP connection using XMPP-over-BOSH (which is a long polling technique), rabbitmq has a Stomp plugin and there are probably other HTTP interfaces I don't know of, and so on. But in general, if you're not being forced to use HTTP, then as the other posters have mentioned it might be worth considering the alternatives. From ericbmerritt@REDACTED Wed Jul 13 02:31:32 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 12 Jul 2011 19:31:32 -0500 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: Thats not possible at the moment. Dialyzer will do a good job at compile time warning you that something is wrong. I think thats about your best bet. Now, as I understand it this type information *is* retained in the compiled beam file. So you could possibly write something that will check the types for you, but I don't think it would be very performant. I think the right answer, at least with out more details, is to do a good job declaring specs and using them, while relying on dialyzer to tell you when there are problems. On Tue, Jul 12, 2011 at 6:26 PM, Tristan Sloughter wrote: > I'm pretty sure this isn't possible but I wanted to be sure before working > around the lack of it. > if I have a type and record like: > -type password() :: binary(). > -record(user, {username :: binary(), > ? ? ? ? ? ? ? ? ? ? password ?:: password()}. > At run time there is no way to tell a field is suppose to be of some type > during runtime, right? > Basically I'd like to be able to have generic or generated create functions > for records that may have to modify arguments (like a password has to be > encrypted from the string provided) when creating the record. So the create > function would pass each field, its value and its type to a transform > function that matches on type and returns the?appropriately?modified value. > Any ideas on how to do this with type definitions or do I need to use tuples > and an atom like {Type, Value} for the value of every field to achieve this? > Thanks, > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From tristan.sloughter@REDACTED Wed Jul 13 02:35:24 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 12 Jul 2011 19:35:24 -0500 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: OK, that is what I thought. But declaring specs and relying on dialyzer for problems doesn't help when I want to match on a type. But I guess I'll have to either bundle the "type" with the value of the record field. Or have some module that accepts a record field atom and returns its "type"... Tristan On Tue, Jul 12, 2011 at 7:31 PM, Eric Merritt wrote: > Thats not possible at the moment. Dialyzer will do a good job at > compile time warning you that something is wrong. I think thats about > your best bet. > > Now, as I understand it this type information *is* retained in the > compiled beam file. So you could possibly write something that will > check the types for you, but I don't think it would be very > performant. > > I think the right answer, at least with out more details, is to do a > good job declaring specs and using them, while relying on dialyzer to > tell you when there are problems. > > On Tue, Jul 12, 2011 at 6:26 PM, Tristan Sloughter > wrote: > > I'm pretty sure this isn't possible but I wanted to be sure before > working > > around the lack of it. > > if I have a type and record like: > > -type password() :: binary(). > > -record(user, {username :: binary(), > > password :: password()}. > > At run time there is no way to tell a field is suppose to be of some type > > during runtime, right? > > Basically I'd like to be able to have generic or generated create > functions > > for records that may have to modify arguments (like a password has to be > > encrypted from the string provided) when creating the record. So the > create > > function would pass each field, its value and its type to a transform > > function that matches on type and returns the appropriately modified > value. > > Any ideas on how to do this with type definitions or do I need to use > tuples > > and an atom like {Type, Value} for the value of every field to achieve > this? > > Thanks, > > Tristan > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Wed Jul 13 03:03:58 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 12 Jul 2011 20:03:58 -0500 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: Maybe its worth parsing at initialization and storing somewhere then, better to define the types once in the spec and reuse them then force the dev to define them multiple times and keep them in sync. Whether you can do that or not will depend very much on the system you are writing of course. On Tue, Jul 12, 2011 at 7:35 PM, Tristan Sloughter wrote: > OK, that is what I thought. But declaring specs and relying on dialyzer for > problems doesn't help when I want to match on a type. But I guess I'll have > to either bundle the "type" with the value of the record field. Or have some > module that accepts a record field atom and returns its "type"... > Tristan > > On Tue, Jul 12, 2011 at 7:31 PM, Eric Merritt > wrote: >> >> Thats not possible at the moment. Dialyzer will do a good job at >> compile time warning you that something is wrong. I think thats about >> your best bet. >> >> Now, as I understand it this type information *is* retained in the >> compiled beam file. So you could possibly write something that will >> check the types for you, but I don't think it would be very >> performant. >> >> I think the right answer, at least with out more details, is to do a >> good job declaring specs and using them, while relying on dialyzer to >> tell you when there are problems. >> >> On Tue, Jul 12, 2011 at 6:26 PM, Tristan Sloughter >> wrote: >> > I'm pretty sure this isn't possible but I wanted to be sure before >> > working >> > around the lack of it. >> > if I have a type and record like: >> > -type password() :: binary(). >> > -record(user, {username :: binary(), >> > ? ? ? ? ? ? ? ? ? ? password ?:: password()}. >> > At run time there is no way to tell a field is suppose to be of some >> > type >> > during runtime, right? >> > Basically I'd like to be able to have generic or generated create >> > functions >> > for records that may have to modify arguments (like a password has to be >> > encrypted from the string provided) when creating the record. So the >> > create >> > function would pass each field, its value and its type to a transform >> > function that matches on type and returns the?appropriately?modified >> > value. >> > Any ideas on how to do this with type definitions or do I need to use >> > tuples >> > and an atom like {Type, Value} for the value of every field to achieve >> > this? >> > Thanks, >> > Tristan >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > From carlsson.richard@REDACTED Wed Jul 13 08:49:17 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 13 Jul 2011 08:49:17 +0200 Subject: [erlang-questions] edoc: documenting "-type" In-Reply-To: References: Message-ID: <4E1D3FED.4070409@gmail.com> On 2011-07-12 23:25, Daniel Goertzen wrote: > I am trying to document a "-type" close to the top of my modules like this... > > %% @doc This type is awesome. Really, it is great. > -type key() :: binary() | atom(). > > ... but it collides with my module @doc entry above it ("multiple @doc > tag" error). > > Is there a right way to document types or is this not supported right > now? The generated documentation shows all my types nicely, but I'd > love to have a bit of explanation appear with them. It's not supported right now, but it's a good idea. Thanks. /Richard From erlang@REDACTED Wed Jul 13 09:16:25 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 13 Jul 2011 09:16:25 +0200 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: On Tue, Jul 12, 2011 at 11:03 PM, Ale wrote: > Hi, > > Just a thought here... Erlang is great for scalability and handling > multiple connections, and well I'm sure a lot of people can give you a > better description fo this, but it occurred to me if you considered > other protocols instead of HTTP? YES - YES - YES Listen carefully - radio bandwidth is *extremely* limited - if you are making an andriod application you should weigh each byte on golden scales. At the moment the demand curve for radio bandwidth is far steeper than the supply curve. We are running out of bandwidth fast - which is why I can't read aftonbladet on the underground any more - the first page takes longer than the journey time. If I was the only person with a smart phone that would be fine - but I'm not. Even the low-bandwidth text applications crawl along when in on the underground. All of this (looking at aftonbladet etc) when I'm out of the congested zones. We (Ericsson) are churning out radio base stations as fast as we can manufacture them, the demand is insatiable. As people move to smart phones they use c. 4 x the bandwidth they used prior to smartphones. Within one 3G cell there is a total of c. 2.4 MBits/second that must be shared by *all* users in the cell. bandwidth is not unlimited it is a finite and limited resource. 4G makes life better with better protocols and encoding methods. Bandwidth (ie the 2.4 MBits) is a limited *shared* resource - this not fibre - this is a limitation reflecting the laws of physics and has nothing to do with engineering. Since bandwidth is a limited resource people will get really pissed off when they get bad response times. As the 3G network get congested they drop back to GRPS/edge etc and do packet switching which places even more load on the networks. So the rule for mobile apps is Cache as much as possible - send and receive as little as possible and weigh each sent and received bytes on golden scales. Bandwidth and bytes sent/received costs money. Most accounts cap data sent/month so even though you have unlimited access they may well cut you back to 64KB/s after you have sent the first Gbits of data. If I were building an adriod app I'd do as follows: 1) Cache as much as possible when bandwidth is high and cheap Most phones have dual access - when they come into WiFi range pro-actively cache as much as possible If you assume that the cost of sending data is a few hundred times the cost of caching the data then you won't go far wrong 2) When you get congested weigh every byte on golden scales. Do not send HTML with all those unnecessary headers: Be proactive - It will be years before the standards change and HTML is dumped for radio access - the good reactive applications will use internal non-standard protocols. /Joe > The problem seems to be there rather > than the language you write your sever. It occurred to me that you > might benefit from using Jabber/XMPP. Googling I found some iOS jabber > clients http://code.google.com/p/xmppframework/ and the cannonical > jabber server is written in Erlang. > > Regards, > > 2011/7/12 Matthew Hillsborough : >> Greetings Erlang community, >> Let me further elaborate on my question that's in the subject of this >> message. I tried to reach out with this question on StackOverflow, however I >> did not have much luck there. Perhaps the community here can provide some >> feedback here for me to let me know if I'm on the right track or if Erlang >> is not the right tool for what I'm trying to accomplish. >> >> I'm building native mobile applications in both iOS and Android. These apps >> require "realtime" updates from and to the server, same as many (but not >> all) network-based application does (Facebook, social games like Words with >> Friends, Finance applications, etc). The communication here is >> bi-directional, in the sense that the server might have updates for the >> mobile clients and the clients will be pushing data down to the server >> whenever necessary. >> >> I think using HTTP long polling for this is over kill in the sense that long >> polling can be detrimental to battery life, especially with a lot of TCP >> setup/teardown for every HTTP connection the device needs to send out >> through the wire. It might make sense to have the mobile applications use >> persistent TCP sockets to establish a connection to the server, and send RPC >> style commands to the server for all web service communication. This >> ofcourse, would require a server to handle the long-lived TCP connection and >> be able to speak to a web service once it makes sense of the data passed >> down the TCP pipe. I'm thinking of passing data in plain text using JSON or >> XML and then using some kind of Erlang interface to HTTP to call a web >> service to handle all the REST type communication. The responses would then >> go back to the "RPC" Erlang instance, which would send the updates to the >> appropriate client(s). >> >> Perhaps an Erlang based RPC server would do well for a network based >> application like this. It would allow for the mobile apps to send and >> receive data from the server all over one connection without multiple >> setup/tear down that individual HTTP requests would do. Since no web browser >> is involved, we do not need to deal with the nuances of HTTP long-polling at >> the mobile client level. I also haven't seen great long polling/keep-alive >> support on the client-side in iOS, but that's irrelevant for the community >> here. >> >> A lot of these "COMET" and long-polling/streaming servers are built with >> HTTP in mind. I'm thinking just using a plain-text protocol over TCP is >> better catered for the type of app I'm building, will make the client more >> responsive, allow for receiving of updates from the server without >> constantly polling the server, etc. >> >> I also looked into HTTP pipelining, but it doesn't look to be worth the >> trouble when it comes to implementing it on the clients. Also, I'm not sure >> if it would allow for bi-directional communication in the client<->server >> communication channel. >> >> Am I completely out of line in thinking that building a custom solution in >> Erlang is a good idea here? To my understanding, Erlang excels at servers >> like this, and if I run the server on tcp/80, I should be able to avoid most >> firewall/port issues. The client would need work to deal with timeouts, re >> connections, acknowledging receipt of asynchronous requests, but that's not >> Erlang's problem. >> >> Has anyone built something similar before? Should I just stick to a web >> server and deal with "COMET" type technologies? (WebSockets, long-polling, >> client-side polling). >> >> Was hoping someone could solidify that I'm not entirely insane for wanting a >> better solution than HTTP would serve in this case, at least at the client >> level. I'll still be using HTTP/REST extensively, the Erlang server would >> just handle the persistent connections and messaging to the Web Service >> (which would probably be something like Django or Rails). >> >> Sorry for the long post; I am just excited to get into the heads of people >> who are smarter than I. >> >> Happy hacking! >> >> Matthew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > > -- > Ale. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Wed Jul 13 09:39:20 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 13 Jul 2011 09:39:20 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: <4E1D4BA8.5070602@dev-extend.eu> On 07/12/2011 10:01 PM, Joe Armstrong wrote: > 90% of time = understanding the problem > 10% coding the final version > > (in the 90% I'll write lots of code to throw away - no point in having > a revision control system for this bit) - I don't want to keep it This part is very important. That's how good code is written. It is essential however to write code in the 90% of time, you can't just "think" about it, you have to try an implementation, learn from the mistakes you made, try another, and so on, until you find the best one. First time you might have to rewrite it all, then as you get closer to your goal you get less and less to rewrite. Of course most people with deadlines just stick with the first version of their code. But that's a mistake as it'll leave bugs that you'll have to fix, and the bad code you wrote for the first deadline is going to make the next deadlines harder to get. So you write code that becomes worse as time goes on and the project becomes a mess. If you can't use Joe's method when starting a project and still meet the deadline, then the planning was wrong and you simply needed more time. Don't kill your projects at birth just to meet some arbitrary time requirement. Because I'm sure you all know that estimating development time is far from a science. > Zen development - no tools - just your brain - all we are doing > in making pleasing patterns of zeros and ones that follow the way > of programming. If we follow the way the programs will please us. "Zen" is kind of overused. But that sums it up nicely. Note though that while good tools aren't necessary for development, they are important for maintenance, bug fixing and even bug detection. Bisecting for example allows to find causes of regressions much more efficiently than printf. Tests, dialyzer, git and all the other tools become important only when you are done iterating over your program's architecture. But even then they aren't as important as the code itself. -- Lo?c Hoguin Dev:Extend From gustav.simonsson@REDACTED Wed Jul 13 10:39:15 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Wed, 13 Jul 2011 08:39:15 +0000 (GMT) Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: Message-ID: <1942228649.87771310546355293.JavaMail.root@zimbra> If you feel bold and adventurous, let the client and server communicate with Erlang messages! http://www.burbas.se/2011/04/12/download-erlang-for-ios/ http://www.burbas.se/2010/11/07/erlang-for-android-now-available-for-download/ Cheers, Gustav Simonsson ----- Original Message ----- From: "Matthew Hillsborough" To: erlang-questions@REDACTED Sent: Tuesday, July 12, 2011 10:25:48 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Is Erlang a good tool for this particular project? Greetings Erlang community, Let me further elaborate on my question that's in the subject of this message. I tried to reach out with this question on StackOverflow, however I did not have much luck there. Perhaps the community here can provide some feedback here for me to let me know if I'm on the right track or if Erlang is not the right tool for what I'm trying to accomplish. I'm building native mobile applications in both iOS and Android. These apps require "realtime" updates from and to the server, same as many (but not all) network-based application does (Facebook, social games like Words with Friends, Finance applications, etc). The communication here is bi-directional, in the sense that the server might have updates for the mobile clients and the clients will be pushing data down to the server whenever necessary. I think using HTTP long polling for this is over kill in the sense that long polling can be detrimental to battery life, especially with a lot of TCP setup/teardown for every HTTP connection the device needs to send out through the wire. It might make sense to have the mobile applications use persistent TCP sockets to establish a connection to the server, and send RPC style commands to the server for all web service communication. This ofcourse, would require a server to handle the long-lived TCP connection and be able to speak to a web service once it makes sense of the data passed down the TCP pipe. I'm thinking of passing data in plain text using JSON or XML and then using some kind of Erlang interface to HTTP to call a web service to handle all the REST type communication. The responses would then go back to the "RPC" Erlang instance, which would send the updates to the appropriate client(s). Perhaps an Erlang based RPC server would do well for a network based application like this. It would allow for the mobile apps to send and receive data from the server all over one connection without multiple setup/tear down that individual HTTP requests would do. Since no web browser is involved, we do not need to deal with the nuances of HTTP long-polling at the mobile client level. I also haven't seen great long polling/keep-alive support on the client-side in iOS, but that's irrelevant for the community here. A lot of these "COMET" and long-polling/streaming servers are built with HTTP in mind. I'm thinking just using a plain-text protocol over TCP is better catered for the type of app I'm building, will make the client more responsive, allow for receiving of updates from the server without constantly polling the server, etc. I also looked into HTTP pipelining, but it doesn't look to be worth the trouble when it comes to implementing it on the clients. Also, I'm not sure if it would allow for bi-directional communication in the client<->server communication channel. Am I completely out of line in thinking that building a custom solution in Erlang is a good idea here? To my understanding, Erlang excels at servers like this, and if I run the server on tcp/80, I should be able to avoid most firewall/port issues. The client would need work to deal with timeouts, re connections, acknowledging receipt of asynchronous requests, but that's not Erlang's problem. Has anyone built something similar before? Should I just stick to a web server and deal with "COMET" type technologies? (WebSockets, long-polling, client-side polling). Was hoping someone could solidify that I'm not entirely insane for wanting a better solution than HTTP would serve in this case, at least at the client level. I'll still be using HTTP/REST extensively, the Erlang server would just handle the persistent connections and messaging to the Web Service (which would probably be something like Django or Rails). Sorry for the long post; I am just excited to get into the heads of people who are smarter than I. Happy hacking! Matthew _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Jul 13 11:18:57 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 13 Jul 2011 11:18:57 +0200 Subject: [erlang-questions] Ways to get started In-Reply-To: <4E1D4BA8.5070602@dev-extend.eu> References: <1678534278.503320.1310406069614.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> <4E1D4BA8.5070602@dev-extend.eu> Message-ID: On Wed, Jul 13, 2011 at 9:39 AM, Lo?c Hoguin wrote: > On 07/12/2011 10:01 PM, Joe Armstrong wrote: >> ? ? 90% of time = understanding the problem >> ? ? 10% coding the final version >> >> (in the 90% I'll write lots of code to throw away - no point in having >> a revision control system for this bit) - I don't want to keep it > > This part is very important. That's how good code is written. It is > essential however to write code in the 90% of time, you can't just > "think" about it, you have to try an implementation, learn from the > mistakes you made, try another, and so on, until you find the best one. > First time you might have to rewrite it all, then as you get closer to > your goal you get less and less to rewrite. > > Of course most people with deadlines just stick with the first version > of their code. But that's a mistake as it'll leave bugs that you'll have > to fix, and the bad code you wrote for the first deadline is going to > make the next deadlines harder to get. So you write code that becomes > worse as time goes on and the project becomes a mess. > > If you can't use Joe's method when starting a project and still meet the > deadline, then the planning was wrong and you simply needed more time. > Don't kill your projects at birth just to meet some arbitrary time > requirement. Because I'm sure you all know that estimating development > time is far from a science. > >> Zen development - no tools - just your brain - all we are doing >> in making pleasing patterns of zeros and ones that follow the way >> of programming. If we follow the way the programs will please us. > > "Zen" is kind of overused. But that sums it up nicely. > > Note though that while good tools aren't necessary for development, they > are important for maintenance, bug fixing and even bug detection. > Bisecting for example allows to find causes of regressions much more > efficiently than printf. Absolutly - when a team of programmers in volved you need all these things. But this thread was called "Ways to get started". I suggested in the beginning writing a three line program and typing two lines into the shell. That's how to get started. Actually in my book I don't get to storing modules in files until chapter three. So we could omit the three line program. So here's the short version of how to get started: 1) Install erlang (read the FAQ to do this) 2) type erl - you should see a prompt like this > 3) type a command and end it with a dot example: > 1234 + 34535. 35769 REPL systejs are brilliant since the barrier to entry is very low. Just remember the "." at the end of every line :-) That's how to get started - telling people that you need to set up a complex tool chain before you can do this is nonsense. This comes far later, and anyway setting up GIT or whatever has nothing to do with Erlang. /Joe > > Tests, dialyzer, git and all the other tools become important only when > you are done iterating over your program's architecture. But even then > they aren't as important as the code itself. > > -- > Lo?c Hoguin > Dev:Extend > From garazdawi@REDACTED Wed Jul 13 11:30:43 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Wed, 13 Jul 2011 11:30:43 +0200 Subject: [erlang-questions] edoc: documenting "-type" In-Reply-To: <4E1D3FED.4070409@gmail.com> References: <4E1D3FED.4070409@gmail.com> Message-ID: Hi, It is not supported via the traditional edoc syntax, however it is supported by the new type documentation generation. If you write -type key() :: binary() | atom(). %% This type is awesome. Really, it is great. The comment below the type will be included in the generated edoc. Lukas On Wed, Jul 13, 2011 at 8:49 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 2011-07-12 23:25, Daniel Goertzen wrote: > >> I am trying to document a "-type" close to the top of my modules like >> this... >> >> %% @doc This type is awesome. Really, it is great. >> -type key() :: binary() | atom(). >> >> ... but it collides with my module @doc entry above it ("multiple @doc >> tag" error). >> >> Is there a right way to document types or is this not supported right >> now? The generated documentation shows all my types nicely, but I'd >> love to have a bit of explanation appear with them. >> > > It's not supported right now, but it's a good idea. Thanks. > > /Richard > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Wed Jul 13 11:42:01 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 13 Jul 2011 11:42:01 +0200 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: For the basic types you can you a when guard in the function clauses (or case clauses): my_magic(#user{password=PW}) when is_binary(PW) -> ... But that is probably not enough for you. Cheers, Torben On Wed, Jul 13, 2011 at 01:26, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > I'm pretty sure this isn't possible but I wanted to be sure before working > around the lack of it. > > if I have a type and record like: > > -type password() :: binary(). > > -record(user, {username :: binary(), > password :: password()}. > > At run time there is no way to tell a field is suppose to be of some type > during runtime, right? > > Basically I'd like to be able to have generic or generated create functions > for records that may have to modify arguments (like a password has to be > encrypted from the string provided) when creating the record. So the create > function would pass each field, its value and its type to a transform > function that matches on type and returns the appropriately modified value. > > Any ideas on how to do this with type definitions or do I need to use > tuples and an atom like {Type, Value} for the value of every field to > achieve this? > > Thanks, > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From sumasai.shivaprasad@REDACTED Wed Jul 13 14:21:56 2011 From: sumasai.shivaprasad@REDACTED (Suma Shivaprasad) Date: Wed, 13 Jul 2011 17:51:56 +0530 Subject: [erlang-questions] Http client sending http response to wrong handler In-Reply-To: References: Message-ID: Hi, Was busy with some other stuff and could not work on this earlier. Tried with inets - 5.6 latest version as well and see the same problem there as well. Now I am able to isolate the cause for this issue. There seems to be a bug with the timeout handling code in http client. Will try to explain the issue - 1. Lets say we send request R1 at time T1 2. At time T2 = T1 + 60 secs, R1 errors out with {error, timeout} from http client 3. At time T3 from the same process we send Request R2 . 4. At time T4 we get a response for R2 with the R1's request id and R1's response data/headers which was supposed to have timed out at T2 and cleared from the request table. I could figure this out from the Request ids given by http client for R1 and R2. This is causing a cascading failure of all subsequent requests from the same process. All requests after a timeout from http client are failing Can you pls let me know if I can work around this by trying out something on my end or does it require a fix on http client? Thanks Suma On Tue, Jun 7, 2011 at 9:19 PM, Ingela Andin wrote: > Hi! > > Do you still get this using inets-5.6? In that case we must > investigate this further? > In such case do you have a minimal example that will reproduce the error? > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2011/6/6 Suma Shivaprasad : > > Our app makes a lot of HTTP requests and we are facing this issue with > both > > inets-5.5.1 and 5.3.2. > > > > Basically our receive clause for the response is trying to match the > request > > id returned in httpc:request call > > and we see that the match always fails with the wrong Request Id . > > > > We gave seen this mismatch in all 3 receive clauses for > > stream_start, stream and stream_end > > > > What we observed after a lot of trial and error is that if the same pid > > makes the http requests , the responses get kind of muddled up but if we > > spawn a separate process for the httpc:request , it is better . But here > > too we have seen some occurrences of this faulty behaviour. > > > > Has anyone faced this issue ? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.hillsborough@REDACTED Wed Jul 13 14:42:27 2011 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Wed, 13 Jul 2011 08:42:27 -0400 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: Hi Ale and everyone else who replied, That's exactly the point, I don't think HTTP is necessary at all! There is the overhead of sending extraneous HTTP headers over the wire. All of those additional bits take additional CPU time and bandwidth (on a mobile device with limited CPU and even more limited bandwidth!). I see absolutely no need for sending HTTP headers and parsing them from the response for this, particularly because I am not building a web browser based application. I have access to C/C++/Objective-C (on iOS) and Java (on Android) and these are perfectly capable of working with TCP sockets. A friend of mine suggested that I just pass messages to the server as JSON using a prefixed header that specifies the length of the message. That would be it! Simple and compact. I just wanted to validate the idea and see if others thing I'll run into too many edge cases uses TCP sockets via an RPC type server to build this server/client architecture out. I want to really make it reusable so all of my mobile products can use it. Loving all the advice coming in! Thanks all. Matthew. On Tue, Jul 12, 2011 at 5:03 PM, Ale wrote: > Hi, > > Just a thought here... Erlang is great for scalability and handling > multiple connections, and well I'm sure a lot of people can give you a > better description fo this, but it occurred to me if you considered > other protocols instead of HTTP? The problem seems to be there rather > than the language you write your sever. It occurred to me that you > might benefit from using Jabber/XMPP. Googling I found some iOS jabber > clients http://code.google.com/p/xmppframework/ and the cannonical > jabber server is written in Erlang. > > Regards, > > 2011/7/12 Matthew Hillsborough : > > Greetings Erlang community, > > Let me further elaborate on my question that's in the subject of this > > message. I tried to reach out with this question on StackOverflow, > however I > > did not have much luck there. Perhaps the community here can provide some > > feedback here for me to let me know if I'm on the right track or if > Erlang > > is not the right tool for what I'm trying to accomplish. > > > > I'm building native mobile applications in both iOS and Android. These > apps > > require "realtime" updates from and to the server, same as many (but not > > all) network-based application does (Facebook, social games like Words > with > > Friends, Finance applications, etc). The communication here is > > bi-directional, in the sense that the server might have updates for the > > mobile clients and the clients will be pushing data down to the server > > whenever necessary. > > > > I think using HTTP long polling for this is over kill in the sense that > long > > polling can be detrimental to battery life, especially with a lot of TCP > > setup/teardown for every HTTP connection the device needs to send out > > through the wire. It might make sense to have the mobile applications use > > persistent TCP sockets to establish a connection to the server, and send > RPC > > style commands to the server for all web service communication. This > > ofcourse, would require a server to handle the long-lived TCP connection > and > > be able to speak to a web service once it makes sense of the data passed > > down the TCP pipe. I'm thinking of passing data in plain text using JSON > or > > XML and then using some kind of Erlang interface to HTTP to call a web > > service to handle all the REST type communication. The responses would > then > > go back to the "RPC" Erlang instance, which would send the updates to the > > appropriate client(s). > > > > Perhaps an Erlang based RPC server would do well for a network based > > application like this. It would allow for the mobile apps to send and > > receive data from the server all over one connection without multiple > > setup/tear down that individual HTTP requests would do. Since no web > browser > > is involved, we do not need to deal with the nuances of HTTP long-polling > at > > the mobile client level. I also haven't seen great long > polling/keep-alive > > support on the client-side in iOS, but that's irrelevant for the > community > > here. > > > > A lot of these "COMET" and long-polling/streaming servers are built with > > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > > better catered for the type of app I'm building, will make the client > more > > responsive, allow for receiving of updates from the server without > > constantly polling the server, etc. > > > > I also looked into HTTP pipelining, but it doesn't look to be worth the > > trouble when it comes to implementing it on the clients. Also, I'm not > sure > > if it would allow for bi-directional communication in the client<->server > > communication channel. > > > > Am I completely out of line in thinking that building a custom solution > in > > Erlang is a good idea here? To my understanding, Erlang excels at servers > > like this, and if I run the server on tcp/80, I should be able to avoid > most > > firewall/port issues. The client would need work to deal with timeouts, > re > > connections, acknowledging receipt of asynchronous requests, but that's > not > > Erlang's problem. > > > > Has anyone built something similar before? Should I just stick to a web > > server and deal with "COMET" type technologies? (WebSockets, > long-polling, > > client-side polling). > > > > Was hoping someone could solidify that I'm not entirely insane for > wanting a > > better solution than HTTP would serve in this case, at least at the > client > > level. I'll still be using HTTP/REST extensively, the Erlang server would > > just handle the persistent connections and messaging to the Web Service > > (which would probably be something like Django or Rails). > > > > Sorry for the long post; I am just excited to get into the heads of > people > > who are smarter than I. > > > > Happy hacking! > > > > Matthew > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Ale. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Wed Jul 13 16:07:46 2011 From: norton@REDACTED (Joseph Norton) Date: Wed, 13 Jul 2011 23:07:46 +0900 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: Matt - There is already a client and server implementation similar to your idea that uses JSON as the wire format over TCP/IP. See the section "JSF" for a brief description: http://norton.github.com/ubf/ubf-user-guide.en.html#_tcp_ip Download and build instructions are here: https://github.com/norton/ubf-jsonrpc Hope you find it helpful. - Joe N. On Jul 13, 2011, at 9:42 PM, Matthew Hillsborough wrote: > Hi Ale and everyone else who replied, > > That's exactly the point, I don't think HTTP is necessary at all! There is the overhead of sending extraneous HTTP headers over the wire. All of those additional bits take additional CPU time and bandwidth (on a mobile device with limited CPU and even more limited bandwidth!). I see absolutely no need for sending HTTP headers and parsing them from the response for this, particularly because I am not building a web browser based application. I have access to C/C++/Objective-C (on iOS) and Java (on Android) and these are perfectly capable of working with TCP sockets. A friend of mine suggested that I just pass messages to the server as JSON using a prefixed header that specifies the length of the message. That would be it! Simple and compact. > > I just wanted to validate the idea and see if others thing I'll run into too many edge cases uses TCP sockets via an RPC type server to build this server/client architecture out. I want to really make it reusable so all of my mobile products can use it. > > Loving all the advice coming in! Thanks all. > > Matthew. > > On Tue, Jul 12, 2011 at 5:03 PM, Ale wrote: > Hi, > > Just a thought here... Erlang is great for scalability and handling > multiple connections, and well I'm sure a lot of people can give you a > better description fo this, but it occurred to me if you considered > other protocols instead of HTTP? The problem seems to be there rather > than the language you write your sever. It occurred to me that you > might benefit from using Jabber/XMPP. Googling I found some iOS jabber > clients http://code.google.com/p/xmppframework/ and the cannonical > jabber server is written in Erlang. > > Regards, > > 2011/7/12 Matthew Hillsborough : > > Greetings Erlang community, > > Let me further elaborate on my question that's in the subject of this > > message. I tried to reach out with this question on StackOverflow, however I > > did not have much luck there. Perhaps the community here can provide some > > feedback here for me to let me know if I'm on the right track or if Erlang > > is not the right tool for what I'm trying to accomplish. > > > > I'm building native mobile applications in both iOS and Android. These apps > > require "realtime" updates from and to the server, same as many (but not > > all) network-based application does (Facebook, social games like Words with > > Friends, Finance applications, etc). The communication here is > > bi-directional, in the sense that the server might have updates for the > > mobile clients and the clients will be pushing data down to the server > > whenever necessary. > > > > I think using HTTP long polling for this is over kill in the sense that long > > polling can be detrimental to battery life, especially with a lot of TCP > > setup/teardown for every HTTP connection the device needs to send out > > through the wire. It might make sense to have the mobile applications use > > persistent TCP sockets to establish a connection to the server, and send RPC > > style commands to the server for all web service communication. This > > ofcourse, would require a server to handle the long-lived TCP connection and > > be able to speak to a web service once it makes sense of the data passed > > down the TCP pipe. I'm thinking of passing data in plain text using JSON or > > XML and then using some kind of Erlang interface to HTTP to call a web > > service to handle all the REST type communication. The responses would then > > go back to the "RPC" Erlang instance, which would send the updates to the > > appropriate client(s). > > > > Perhaps an Erlang based RPC server would do well for a network based > > application like this. It would allow for the mobile apps to send and > > receive data from the server all over one connection without multiple > > setup/tear down that individual HTTP requests would do. Since no web browser > > is involved, we do not need to deal with the nuances of HTTP long-polling at > > the mobile client level. I also haven't seen great long polling/keep-alive > > support on the client-side in iOS, but that's irrelevant for the community > > here. > > > > A lot of these "COMET" and long-polling/streaming servers are built with > > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > > better catered for the type of app I'm building, will make the client more > > responsive, allow for receiving of updates from the server without > > constantly polling the server, etc. > > > > I also looked into HTTP pipelining, but it doesn't look to be worth the > > trouble when it comes to implementing it on the clients. Also, I'm not sure > > if it would allow for bi-directional communication in the client<->server > > communication channel. > > > > Am I completely out of line in thinking that building a custom solution in > > Erlang is a good idea here? To my understanding, Erlang excels at servers > > like this, and if I run the server on tcp/80, I should be able to avoid most > > firewall/port issues. The client would need work to deal with timeouts, re > > connections, acknowledging receipt of asynchronous requests, but that's not > > Erlang's problem. > > > > Has anyone built something similar before? Should I just stick to a web > > server and deal with "COMET" type technologies? (WebSockets, long-polling, > > client-side polling). > > > > Was hoping someone could solidify that I'm not entirely insane for wanting a > > better solution than HTTP would serve in this case, at least at the client > > level. I'll still be using HTTP/REST extensively, the Erlang server would > > just handle the persistent connections and messaging to the Web Service > > (which would probably be something like Django or Rails). > > > > Sorry for the long post; I am just excited to get into the heads of people > > who are smarter than I. > > > > Happy hacking! > > > > Matthew > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > Ale. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Jul 13 16:13:19 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 13 Jul 2011 09:13:19 -0500 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: Yeah, I was hoping for something that would work with user defined typed. I assume parse transforms when parsing a record have the type information? If so I think I can use that to generate the necessary functionality I'm looking for. Tristan On Wed, Jul 13, 2011 at 4:42 AM, Torben Hoffmann wrote: > For the basic types you can you a when guard in the function clauses (or > case clauses): > > my_magic(#user{password=PW}) when is_binary(PW) -> ... > > But that is probably not enough for you. > > Cheers, > Torben > > On Wed, Jul 13, 2011 at 01:26, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> I'm pretty sure this isn't possible but I wanted to be sure before working >> around the lack of it. >> >> if I have a type and record like: >> >> -type password() :: binary(). >> >> -record(user, {username :: binary(), >> password :: password()}. >> >> At run time there is no way to tell a field is suppose to be of some type >> during runtime, right? >> >> Basically I'd like to be able to have generic or generated create >> functions for records that may have to modify arguments (like a password has >> to be encrypted from the string provided) when creating the record. So the >> create function would pass each field, its value and its type to a transform >> function that matches on type and returns the appropriately modified value. >> >> Any ideas on how to do this with type definitions or do I need to use >> tuples and an atom like {Type, Value} for the value of every field to >> achieve this? >> >> Thanks, >> Tristan >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > http://www.linkedin.com/in/torbenhoffmann > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.hillsborough@REDACTED Wed Jul 13 16:16:59 2011 From: matthew.hillsborough@REDACTED (Matthew Hillsborough) Date: Wed, 13 Jul 2011 10:16:59 -0400 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: <9486E9A0-9531-4A68-A32A-9EA4D72BCA9F@alum.mit.edu> References: <9486E9A0-9531-4A68-A32A-9EA4D72BCA9F@alum.mit.edu> Message-ID: Hi Joe, Thanks for posting links to this. Surprised Joe A. did not mention it in his post, it has him listed as an author. :) Just from a quick glimpse, this appears to be using MochiWeb, which is completely HTTP based. I am not entirely sure if that's the route I am looking to go, unless I am overlooking something here. With that said, there are probably parts of this code I can pick out and re-use in a pure TCP socket implementation (if this is indeed purely HTTP, which I suspect it might be). Would love to hear I am wrong though so I do not need to reinvent the wheel. Thank you and happy hacking - Matthew On Wed, Jul 13, 2011 at 9:43 AM, Joseph Norton wrote: > > Matt - > > There is already a client and server implementation similar to your idea > that uses JSON as the wire format over TCP/IP. See the section "JSF" for a > brief description: > > http://norton.github.com/ubf/ubf-user-guide.en.html#_tcp_ip > > Download and building instructions are here: > > https://github.com/norton/ubf-jsonrpc > > Hope you find it helpful. > > - Joe N. > > > On Jul 13, 2011, at 9:42 PM, Matthew Hillsborough wrote: > > Hi Ale and everyone else who replied, > > That's exactly the point, I don't think HTTP is necessary at all! There is > the overhead of sending extraneous HTTP headers over the wire. All of those > additional bits take additional CPU time and bandwidth (on a mobile device > with limited CPU and even more limited bandwidth!). I see absolutely no need > for sending HTTP headers and parsing them from the response for this, > particularly because I am not building a web browser based application. I > have access to C/C++/Objective-C (on iOS) and Java (on Android) and these > are perfectly capable of working with TCP sockets. A friend of mine > suggested that I just pass messages to the server as JSON using a prefixed > header that specifies the length of the message. That would be it! Simple > and compact. > > I just wanted to validate the idea and see if others thing I'll run into > too many edge cases uses TCP sockets via an RPC type server to build this > server/client architecture out. I want to really make it reusable so all of > my mobile products can use it. > > Loving all the advice coming in! Thanks all. > > Matthew. > > On Tue, Jul 12, 2011 at 5:03 PM, Ale wrote: > >> Hi, >> >> Just a thought here... Erlang is great for scalability and handling >> multiple connections, and well I'm sure a lot of people can give you a >> better description fo this, but it occurred to me if you considered >> other protocols instead of HTTP? The problem seems to be there rather >> than the language you write your sever. It occurred to me that you >> might benefit from using Jabber/XMPP. Googling I found some iOS jabber >> clients http://code.google.com/p/xmppframework/ and the cannonical >> jabber server is written in Erlang. >> >> Regards, >> >> 2011/7/12 Matthew Hillsborough : >> > Greetings Erlang community, >> > Let me further elaborate on my question that's in the subject of this >> > message. I tried to reach out with this question on StackOverflow, >> however I >> > did not have much luck there. Perhaps the community here can provide >> some >> > feedback here for me to let me know if I'm on the right track or if >> Erlang >> > is not the right tool for what I'm trying to accomplish. >> > >> > I'm building native mobile applications in both iOS and Android. These >> apps >> > require "realtime" updates from and to the server, same as many (but not >> > all) network-based application does (Facebook, social games like Words >> with >> > Friends, Finance applications, etc). The communication here is >> > bi-directional, in the sense that the server might have updates for the >> > mobile clients and the clients will be pushing data down to the server >> > whenever necessary. >> > >> > I think using HTTP long polling for this is over kill in the sense that >> long >> > polling can be detrimental to battery life, especially with a lot of TCP >> > setup/teardown for every HTTP connection the device needs to send out >> > through the wire. It might make sense to have the mobile applications >> use >> > persistent TCP sockets to establish a connection to the server, and send >> RPC >> > style commands to the server for all web service communication. This >> > ofcourse, would require a server to handle the long-lived TCP connection >> and >> > be able to speak to a web service once it makes sense of the data passed >> > down the TCP pipe. I'm thinking of passing data in plain text using JSON >> or >> > XML and then using some kind of Erlang interface to HTTP to call a web >> > service to handle all the REST type communication. The responses would >> then >> > go back to the "RPC" Erlang instance, which would send the updates to >> the >> > appropriate client(s). >> > >> > Perhaps an Erlang based RPC server would do well for a network based >> > application like this. It would allow for the mobile apps to send and >> > receive data from the server all over one connection without multiple >> > setup/tear down that individual HTTP requests would do. Since no web >> browser >> > is involved, we do not need to deal with the nuances of HTTP >> long-polling at >> > the mobile client level. I also haven't seen great long >> polling/keep-alive >> > support on the client-side in iOS, but that's irrelevant for the >> community >> > here. >> > >> > A lot of these "COMET" and long-polling/streaming servers are built with >> > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is >> > better catered for the type of app I'm building, will make the client >> more >> > responsive, allow for receiving of updates from the server without >> > constantly polling the server, etc. >> > >> > I also looked into HTTP pipelining, but it doesn't look to be worth the >> > trouble when it comes to implementing it on the clients. Also, I'm not >> sure >> > if it would allow for bi-directional communication in the >> client<->server >> > communication channel. >> > >> > Am I completely out of line in thinking that building a custom solution >> in >> > Erlang is a good idea here? To my understanding, Erlang excels at >> servers >> > like this, and if I run the server on tcp/80, I should be able to avoid >> most >> > firewall/port issues. The client would need work to deal with timeouts, >> re >> > connections, acknowledging receipt of asynchronous requests, but that's >> not >> > Erlang's problem. >> > >> > Has anyone built something similar before? Should I just stick to a web >> > server and deal with "COMET" type technologies? (WebSockets, >> long-polling, >> > client-side polling). >> > >> > Was hoping someone could solidify that I'm not entirely insane for >> wanting a >> > better solution than HTTP would serve in this case, at least at the >> client >> > level. I'll still be using HTTP/REST extensively, the Erlang server >> would >> > just handle the persistent connections and messaging to the Web Service >> > (which would probably be something like Django or Rails). >> > >> > Sorry for the long post; I am just excited to get into the heads of >> people >> > who are smarter than I. >> > >> > Happy hacking! >> > >> > Matthew >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> >> -- >> Ale. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Joseph Norton > norton@REDACTED > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Wed Jul 13 16:26:04 2011 From: norton@REDACTED (Joseph Norton) Date: Wed, 13 Jul 2011 23:26:04 +0900 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: Tristan - It has been awhile since I dabbled with this code - it is an incomplete prototype to re-use a module's types as part of a UBF contract. Nevertheless, there is a working unit test that illustrates using a parse transform on a module's types. The test module is here: https://github.com/norton/ubf-eep8/blob/master/test/eunit/ubf_eep8_samples.erl The unit test is here: https://github.com/norton/ubf-eep8/blob/master/test/eunit/ubf_eep8_samples_tests.erl The parse transform code is here: https://github.com/norton/ubf-eep8/blob/master/src/eep8_contract_parser.erl and finally instructions to download and to build are here: https://github.com/norton/ubf-eep8 Hope this helps. - Joe N. On Jul 13, 2011, at 11:13 PM, Tristan Sloughter wrote: > Yeah, I was hoping for something that would work with user defined typed. > > I assume parse transforms when parsing a record have the type information? If so I think I can use that to generate the necessary functionality I'm looking for. > > Tristan > > On Wed, Jul 13, 2011 at 4:42 AM, Torben Hoffmann wrote: > For the basic types you can you a when guard in the function clauses (or case clauses): > > my_magic(#user{password=PW}) when is_binary(PW) -> ... > > But that is probably not enough for you. > > Cheers, > Torben > > On Wed, Jul 13, 2011 at 01:26, Tristan Sloughter wrote: > I'm pretty sure this isn't possible but I wanted to be sure before working around the lack of it. > > if I have a type and record like: > > -type password() :: binary(). > > -record(user, {username :: binary(), > password :: password()}. > > At run time there is no way to tell a field is suppose to be of some type during runtime, right? > > Basically I'd like to be able to have generic or generated create functions for records that may have to modify arguments (like a password has to be encrypted from the string provided) when creating the record. So the create function would pass each field, its value and its type to a transform function that matches on type and returns the appropriately modified value. > > Any ideas on how to do this with type definitions or do I need to use tuples and an atom like {Type, Value} for the value of every field to achieve this? > > Thanks, > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > http://www.linkedin.com/in/torbenhoffmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Jul 13 16:27:34 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 13 Jul 2011 09:27:34 -0500 Subject: [erlang-questions] Checking Types at Runtime In-Reply-To: References: Message-ID: Great, thanks! This should help a lot. On Wed, Jul 13, 2011 at 9:26 AM, Joseph Norton wrote: > > Tristan - > > It has been awhile since I dabbled with this code - it is an incomplete > prototype to re-use a module's types as part of a UBF contract. > Nevertheless, there is a working unit test that illustrates using a parse > transform on a module's types. > > The test module is here: > > > https://github.com/norton/ubf-eep8/blob/master/test/eunit/ubf_eep8_samples.erl > > The unit test is here: > > > https://github.com/norton/ubf-eep8/blob/master/test/eunit/ubf_eep8_samples_tests.erl > > The parse transform code is here: > > https://github.com/norton/ubf-eep8/blob/master/src/eep8_contract_parser.erl > > and finally instructions to download and to build are here: > > https://github.com/norton/ubf-eep8 > > Hope this helps. > > - Joe N. > > > On Jul 13, 2011, at 11:13 PM, Tristan Sloughter wrote: > > Yeah, I was hoping for something that would work with user defined typed. > > I assume parse transforms when parsing a record have the type information? > If so I think I can use that to generate the necessary functionality I'm > looking for. > > Tristan > > On Wed, Jul 13, 2011 at 4:42 AM, Torben Hoffmann wrote: > >> For the basic types you can you a when guard in the function clauses (or >> case clauses): >> >> my_magic(#user{password=PW}) when is_binary(PW) -> ... >> >> But that is probably not enough for you. >> >> Cheers, >> Torben >> >> On Wed, Jul 13, 2011 at 01:26, Tristan Sloughter < >> tristan.sloughter@REDACTED> wrote: >> >>> I'm pretty sure this isn't possible but I wanted to be sure before >>> working around the lack of it. >>> >>> if I have a type and record like: >>> >>> -type password() :: binary(). >>> >>> -record(user, {username :: binary(), >>> password :: password()}. >>> >>> At run time there is no way to tell a field is suppose to be of some type >>> during runtime, right? >>> >>> Basically I'd like to be able to have generic or generated create >>> functions for records that may have to modify arguments (like a password has >>> to be encrypted from the string provided) when creating the record. So the >>> create function would pass each field, its value and its type to a transform >>> function that matches on type and returns the appropriately modified value. >>> >>> Any ideas on how to do this with type definitions or do I need to use >>> tuples and an atom like {Type, Value} for the value of every field to >>> achieve this? >>> >>> Thanks, >>> Tristan >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> http://www.linkedin.com/in/torbenhoffmann >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Joseph Norton > norton@REDACTED > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Wed Jul 13 16:35:46 2011 From: norton@REDACTED (Joseph Norton) Date: Wed, 13 Jul 2011 23:35:46 +0900 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: <9486E9A0-9531-4A68-A32A-9EA4D72BCA9F@alum.mit.edu> Message-ID: Matt - This particular application is only using Mochiweb's json encoder/decoder. If you have time and interest, please review the UBF user's guide from the beginning. Hopefully, it will be clearer. The TCP/IP wire formats - UBF, EBF, and JSF all work using the same basic client and server implementation. The only difference being the network wire format. If you have other questions, let me know. - Joe On Jul 13, 2011, at 11:16 PM, Matthew Hillsborough wrote: > Hi Joe, > > Thanks for posting links to this. Surprised Joe A. did not mention it in his post, it has him listed as an author. :) > > Just from a quick glimpse, this appears to be using MochiWeb, which is completely HTTP based. I am not entirely sure if that's the route I am looking to go, unless I am overlooking something here. With that said, there are probably parts of this code I can pick out and re-use in a pure TCP socket implementation (if this is indeed purely HTTP, which I suspect it might be). > > Would love to hear I am wrong though so I do not need to reinvent the wheel. > > Thank you and happy hacking - > > Matthew > > On Wed, Jul 13, 2011 at 9:43 AM, Joseph Norton wrote: > > Matt - > > There is already a client and server implementation similar to your idea that uses JSON as the wire format over TCP/IP. See the section "JSF" for a brief description: > > http://norton.github.com/ubf/ubf-user-guide.en.html#_tcp_ip > > Download and building instructions are here: > > https://github.com/norton/ubf-jsonrpc > > Hope you find it helpful. > > - Joe N. > > > On Jul 13, 2011, at 9:42 PM, Matthew Hillsborough wrote: > >> Hi Ale and everyone else who replied, >> >> That's exactly the point, I don't think HTTP is necessary at all! There is the overhead of sending extraneous HTTP headers over the wire. All of those additional bits take additional CPU time and bandwidth (on a mobile device with limited CPU and even more limited bandwidth!). I see absolutely no need for sending HTTP headers and parsing them from the response for this, particularly because I am not building a web browser based application. I have access to C/C++/Objective-C (on iOS) and Java (on Android) and these are perfectly capable of working with TCP sockets. A friend of mine suggested that I just pass messages to the server as JSON using a prefixed header that specifies the length of the message. That would be it! Simple and compact. >> >> I just wanted to validate the idea and see if others thing I'll run into too many edge cases uses TCP sockets via an RPC type server to build this server/client architecture out. I want to really make it reusable so all of my mobile products can use it. >> >> Loving all the advice coming in! Thanks all. >> >> Matthew. >> >> On Tue, Jul 12, 2011 at 5:03 PM, Ale wrote: >> Hi, >> >> Just a thought here... Erlang is great for scalability and handling >> multiple connections, and well I'm sure a lot of people can give you a >> better description fo this, but it occurred to me if you considered >> other protocols instead of HTTP? The problem seems to be there rather >> than the language you write your sever. It occurred to me that you >> might benefit from using Jabber/XMPP. Googling I found some iOS jabber >> clients http://code.google.com/p/xmppframework/ and the cannonical >> jabber server is written in Erlang. >> >> Regards, >> >> 2011/7/12 Matthew Hillsborough : >> > Greetings Erlang community, >> > Let me further elaborate on my question that's in the subject of this >> > message. I tried to reach out with this question on StackOverflow, however I >> > did not have much luck there. Perhaps the community here can provide some >> > feedback here for me to let me know if I'm on the right track or if Erlang >> > is not the right tool for what I'm trying to accomplish. >> > >> > I'm building native mobile applications in both iOS and Android. These apps >> > require "realtime" updates from and to the server, same as many (but not >> > all) network-based application does (Facebook, social games like Words with >> > Friends, Finance applications, etc). The communication here is >> > bi-directional, in the sense that the server might have updates for the >> > mobile clients and the clients will be pushing data down to the server >> > whenever necessary. >> > >> > I think using HTTP long polling for this is over kill in the sense that long >> > polling can be detrimental to battery life, especially with a lot of TCP >> > setup/teardown for every HTTP connection the device needs to send out >> > through the wire. It might make sense to have the mobile applications use >> > persistent TCP sockets to establish a connection to the server, and send RPC >> > style commands to the server for all web service communication. This >> > ofcourse, would require a server to handle the long-lived TCP connection and >> > be able to speak to a web service once it makes sense of the data passed >> > down the TCP pipe. I'm thinking of passing data in plain text using JSON or >> > XML and then using some kind of Erlang interface to HTTP to call a web >> > service to handle all the REST type communication. The responses would then >> > go back to the "RPC" Erlang instance, which would send the updates to the >> > appropriate client(s). >> > >> > Perhaps an Erlang based RPC server would do well for a network based >> > application like this. It would allow for the mobile apps to send and >> > receive data from the server all over one connection without multiple >> > setup/tear down that individual HTTP requests would do. Since no web browser >> > is involved, we do not need to deal with the nuances of HTTP long-polling at >> > the mobile client level. I also haven't seen great long polling/keep-alive >> > support on the client-side in iOS, but that's irrelevant for the community >> > here. >> > >> > A lot of these "COMET" and long-polling/streaming servers are built with >> > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is >> > better catered for the type of app I'm building, will make the client more >> > responsive, allow for receiving of updates from the server without >> > constantly polling the server, etc. >> > >> > I also looked into HTTP pipelining, but it doesn't look to be worth the >> > trouble when it comes to implementing it on the clients. Also, I'm not sure >> > if it would allow for bi-directional communication in the client<->server >> > communication channel. >> > >> > Am I completely out of line in thinking that building a custom solution in >> > Erlang is a good idea here? To my understanding, Erlang excels at servers >> > like this, and if I run the server on tcp/80, I should be able to avoid most >> > firewall/port issues. The client would need work to deal with timeouts, re >> > connections, acknowledging receipt of asynchronous requests, but that's not >> > Erlang's problem. >> > >> > Has anyone built something similar before? Should I just stick to a web >> > server and deal with "COMET" type technologies? (WebSockets, long-polling, >> > client-side polling). >> > >> > Was hoping someone could solidify that I'm not entirely insane for wanting a >> > better solution than HTTP would serve in this case, at least at the client >> > level. I'll still be using HTTP/REST extensively, the Erlang server would >> > just handle the persistent connections and messaging to the Web Service >> > (which would probably be something like Django or Rails). >> > >> > Sorry for the long post; I am just excited to get into the heads of people >> > who are smarter than I. >> > >> > Happy hacking! >> > >> > Matthew >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> >> -- >> Ale. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Joseph Norton > norton@REDACTED > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.meyer@REDACTED Wed Jul 13 20:03:08 2011 From: joel.meyer@REDACTED (Joel Meyer) Date: Wed, 13 Jul 2011 11:03:08 -0700 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: <9486E9A0-9531-4A68-A32A-9EA4D72BCA9F@alum.mit.edu> Message-ID: On Wed, Jul 13, 2011 at 7:16 AM, Matthew Hillsborough < matthew.hillsborough@REDACTED> wrote: > Hi Joe, > > Thanks for posting links to this. Surprised Joe A. did not mention it in > his post, it has him listed as an author. :) > > Just from a quick glimpse, this appears to be using MochiWeb, which is > completely HTTP based. I am not entirely sure if that's the route I am > looking to go, unless I am overlooking something here. With that said, there > are probably parts of this code I can pick out and re-use in a pure TCP > socket implementation (if this is indeed purely HTTP, which I suspect it > might be). > > Would love to hear I am wrong though so I do not need to reinvent the > wheel. > > Thank you and happy hacking - > Out of curiosity, is there a reason you wouldn't use thrift or protocol buffers? Even if you don't use their RPC mechanisms, you could use one of those as the serialized form for message passing (over plain sockets, if you so desire). I think a binary serialization format will still be more compact than JSON and faster to deserialize as well. Cheers, Joel > > Matthew > > On Wed, Jul 13, 2011 at 9:43 AM, Joseph Norton wrote: > >> >> Matt - >> >> There is already a client and server implementation similar to your idea >> that uses JSON as the wire format over TCP/IP. See the section "JSF" for a >> brief description: >> >> http://norton.github.com/ubf/ubf-user-guide.en.html#_tcp_ip >> >> Download and building instructions are here: >> >> https://github.com/norton/ubf-jsonrpc >> >> Hope you find it helpful. >> >> - Joe N. >> >> >> On Jul 13, 2011, at 9:42 PM, Matthew Hillsborough wrote: >> >> Hi Ale and everyone else who replied, >> >> That's exactly the point, I don't think HTTP is necessary at all! There is >> the overhead of sending extraneous HTTP headers over the wire. All of those >> additional bits take additional CPU time and bandwidth (on a mobile device >> with limited CPU and even more limited bandwidth!). I see absolutely no need >> for sending HTTP headers and parsing them from the response for this, >> particularly because I am not building a web browser based application. I >> have access to C/C++/Objective-C (on iOS) and Java (on Android) and these >> are perfectly capable of working with TCP sockets. A friend of mine >> suggested that I just pass messages to the server as JSON using a prefixed >> header that specifies the length of the message. That would be it! Simple >> and compact. >> >> I just wanted to validate the idea and see if others thing I'll run into >> too many edge cases uses TCP sockets via an RPC type server to build this >> server/client architecture out. I want to really make it reusable so all of >> my mobile products can use it. >> >> Loving all the advice coming in! Thanks all. >> >> Matthew. >> >> On Tue, Jul 12, 2011 at 5:03 PM, Ale wrote: >> >>> Hi, >>> >>> Just a thought here... Erlang is great for scalability and handling >>> multiple connections, and well I'm sure a lot of people can give you a >>> better description fo this, but it occurred to me if you considered >>> other protocols instead of HTTP? The problem seems to be there rather >>> than the language you write your sever. It occurred to me that you >>> might benefit from using Jabber/XMPP. Googling I found some iOS jabber >>> clients http://code.google.com/p/xmppframework/ and the cannonical >>> jabber server is written in Erlang. >>> >>> Regards, >>> >>> 2011/7/12 Matthew Hillsborough : >>> > Greetings Erlang community, >>> > Let me further elaborate on my question that's in the subject of this >>> > message. I tried to reach out with this question on StackOverflow, >>> however I >>> > did not have much luck there. Perhaps the community here can provide >>> some >>> > feedback here for me to let me know if I'm on the right track or if >>> Erlang >>> > is not the right tool for what I'm trying to accomplish. >>> > >>> > I'm building native mobile applications in both iOS and Android. These >>> apps >>> > require "realtime" updates from and to the server, same as many (but >>> not >>> > all) network-based application does (Facebook, social games like Words >>> with >>> > Friends, Finance applications, etc). The communication here is >>> > bi-directional, in the sense that the server might have updates for the >>> > mobile clients and the clients will be pushing data down to the server >>> > whenever necessary. >>> > >>> > I think using HTTP long polling for this is over kill in the sense that >>> long >>> > polling can be detrimental to battery life, especially with a lot of >>> TCP >>> > setup/teardown for every HTTP connection the device needs to send out >>> > through the wire. It might make sense to have the mobile applications >>> use >>> > persistent TCP sockets to establish a connection to the server, and >>> send RPC >>> > style commands to the server for all web service communication. This >>> > ofcourse, would require a server to handle the long-lived TCP >>> connection and >>> > be able to speak to a web service once it makes sense of the data >>> passed >>> > down the TCP pipe. I'm thinking of passing data in plain text using >>> JSON or >>> > XML and then using some kind of Erlang interface to HTTP to call a web >>> > service to handle all the REST type communication. The responses would >>> then >>> > go back to the "RPC" Erlang instance, which would send the updates to >>> the >>> > appropriate client(s). >>> > >>> > Perhaps an Erlang based RPC server would do well for a network based >>> > application like this. It would allow for the mobile apps to send and >>> > receive data from the server all over one connection without multiple >>> > setup/tear down that individual HTTP requests would do. Since no web >>> browser >>> > is involved, we do not need to deal with the nuances of HTTP >>> long-polling at >>> > the mobile client level. I also haven't seen great long >>> polling/keep-alive >>> > support on the client-side in iOS, but that's irrelevant for the >>> community >>> > here. >>> > >>> > A lot of these "COMET" and long-polling/streaming servers are built >>> with >>> > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is >>> > better catered for the type of app I'm building, will make the client >>> more >>> > responsive, allow for receiving of updates from the server without >>> > constantly polling the server, etc. >>> > >>> > I also looked into HTTP pipelining, but it doesn't look to be worth the >>> > trouble when it comes to implementing it on the clients. Also, I'm not >>> sure >>> > if it would allow for bi-directional communication in the >>> client<->server >>> > communication channel. >>> > >>> > Am I completely out of line in thinking that building a custom solution >>> in >>> > Erlang is a good idea here? To my understanding, Erlang excels at >>> servers >>> > like this, and if I run the server on tcp/80, I should be able to avoid >>> most >>> > firewall/port issues. The client would need work to deal with timeouts, >>> re >>> > connections, acknowledging receipt of asynchronous requests, but that's >>> not >>> > Erlang's problem. >>> > >>> > Has anyone built something similar before? Should I just stick to a web >>> > server and deal with "COMET" type technologies? (WebSockets, >>> long-polling, >>> > client-side polling). >>> > >>> > Was hoping someone could solidify that I'm not entirely insane for >>> wanting a >>> > better solution than HTTP would serve in this case, at least at the >>> client >>> > level. I'll still be using HTTP/REST extensively, the Erlang server >>> would >>> > just handle the persistent connections and messaging to the Web Service >>> > (which would probably be something like Django or Rails). >>> > >>> > Sorry for the long post; I am just excited to get into the heads of >>> people >>> > who are smarter than I. >>> > >>> > Happy hacking! >>> > >>> > Matthew >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >>> >>> >>> >>> -- >>> Ale. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> Joseph Norton >> norton@REDACTED >> >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Wed Jul 13 21:19:35 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Wed, 13 Jul 2011 22:19:35 +0300 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! Message-ID: Hi All, We're building a nontrivial Erlang application, which will provide a potentially high-volume, public RESTful Web Service. As part of the API, we will be providing both informative text (status, labels, layout templates) and error messages. A twofold question, then, to the experienced Grandmasters in the audience: 1. How have you implemented localization of text? This includes parameterized format strings. 2. Has anyone used composite error values? For example, instead of {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, LocalizedMessageId}, {message_args, [...]}... Should we steer clear of these, and if so, what alternative would you suggest? Thanks in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Thu Jul 14 01:53:15 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 13 Jul 2011 16:53:15 -0700 (PDT) Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: References: Message-ID: <50fffd16-7e4d-4201-9c15-b44a9937a94c@v7g2000vbk.googlegroups.com> Given that "strings" are either ambiguous or implicitly constrained inventions of data representation, then you may consider applying something along the lines of: -record(text, {value, charset = utf8, locale = 'us-en'}). ...to all user-viewable messages. The transformation would use whatever hideously ambiguous or implicitly constrained "string" that bubbles up from infrastructure as the "key" to that tuple value. BTW This problem isn't really an erlang/otp but one that every platform that I have ever investigated. What makes erlang different is that you can easily forsee a potential, and implementable, solution to this legacy issue in CS. /s On Jul 13, 2:19?pm, Alex Arnon wrote: > Hi All, > > We're building a nontrivial Erlang application, which will provide a > potentially high-volume, public RESTful Web Service. As part of the API, we > will be providing both informative text (status, labels, layout templates) > and error messages. A twofold question, then, to the experienced > Grandmasters in the audience: > 1. How have you implemented localization of text? This includes > parameterized format strings. > 2. Has anyone used composite error values? For example, instead of {error, > badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, > LocalizedMessageId}, {message_args, [...]}... Should we steer clear of > these, and if so, what alternative would you suggest? > > Thanks in advance! > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From steven.charles.davis@REDACTED Thu Jul 14 02:01:24 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 13 Jul 2011 17:01:24 -0700 (PDT) Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: <36f673db-7561-4d8d-99e3-110b55a73a69@p14g2000yqj.googlegroups.com> I am tempted to answer the OP with just a simple "Yes". Rationale: Erlang/OTP is a good tool for any project, it has excellent expressiveness, it encourages clarity of thought about a problem, and it is fast to implement appropriate solutions. However, of course, it may not be pandemically the best or most appropriate tool... and I suspect that's the comfort that was requested. /s From jwatte@REDACTED Thu Jul 14 03:10:29 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 13 Jul 2011 18:10:29 -0700 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: My three suggestions were intended to be used together. The HTTP protocol is what defines Basic-Auth. The encapsulation into TLS is transparent to that particular step. My apologies if this confused my recommendation. Logging out can be accomplished by simply sending an "unauthenticated" HTTP status code and a new Authenticate header that tells the client it needs new credentials. HTTP authentication is superior to cookie-based authentication in a number of cases where the REST nature of the web allows for various kinds of mash-ups. Cookie-based authentication really only works well when an interactive user is using a mainline web browser to access your application. Kind-of like using the flash player for your website: some designers think nothing about it; others believe that it significantly reduces the value of the site, at least long-term. Let your requirements decide. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Mon, Jul 11, 2011 at 10:16 AM, Garrett Smith wrote: > On Fri, Jul 8, 2011 at 11:20 PM, Jon Watte wrote: > > > 2) Use Basic-auth over HTTP -- this sends name and password, > > base-64-encoded. > > This is surely a typo. You can't say "HTTP" and expect people to read > "HTTP + TLS". > > For simple web auth, I routinely use basic auth, but only ever over > HTTPS. This doesn't work however if you need to control sessions or > let users log out. It's just a quick and dirty way to control who can > see what. > > Garrett > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jul 14 03:23:41 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 13 Jul 2011 18:23:41 -0700 Subject: [erlang-questions] remsh In-Reply-To: <4E1C7106.6080105@ubiquity.it> References: <4E1C7106.6080105@ubiquity.it> Message-ID: If I understand it correctly, "remsh" specifies to create an interactive shell conneted to the remote node, but a command-line argument is evaluated using the local node. Try evaluating it as a rpc:call expression instead? You won't need a -remsh at all then, instead passing the node argument to the rpc call. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 12, 2011 at 9:06 AM, Carlo Bertoldi wrote: > Hello list, > I'm trying to connect to a remote node and execute a function. > If I do it the manual way, it works: > > erl -sname foo -remsh prova@REDACTED > 1> probe_db_manager:cleanup(). > true > > But if I try to do it entirely from the cli it gets angry: > > erl -sname foo -remsh prova@REDACTED -run probe_db_manager cleanup > > {"init terminating in do_boot",{noproc,{gen_server,** > call,[{global,probe_db_**manager},cleanup]}}} > > probe_db_manager is a gen_server process. > I also tried this: erl -sname foo -remsh prova@REDACTED -run global > whereis_name probe_db_manager > and it returns nothing. > > Thanks, > Carlo > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jul 14 03:25:43 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 13 Jul 2011 18:25:43 -0700 Subject: [erlang-questions] Http client sending http response to wrong handler In-Reply-To: References: Message-ID: I know this is not what you want to hear, but I ended up writing my own http client :-/ Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jul 13, 2011 at 5:21 AM, Suma Shivaprasad < sumasai.shivaprasad@REDACTED> wrote: > Hi, > > Was busy with some other stuff and could not work on this earlier. > Tried with inets - 5.6 latest version as well and see the same problem > there as well. > Now I am able to isolate the cause for this issue. There seems to be a bug > with the timeout handling code in http client. > > Will try to explain the issue - > > 1. Lets say we send request R1 at time T1 > > 2. At time T2 = T1 + 60 secs, R1 errors out with {error, timeout} from > http client > > 3. At time T3 from the same process we send Request R2 . > > 4. At time T4 we get a response for R2 with the R1's request id and R1's > response data/headers which was supposed to have timed out at T2 and > cleared from the request table. I could figure this out from the Request > ids given by http client for R1 and R2. This is causing a cascading failure > of all subsequent requests from the same process. All requests after a > timeout from http client are failing > > Can you pls let me know if I can work around this by trying out something > on my end or does it require a fix on http client? > > Thanks > Suma > > > > > > > > On Tue, Jun 7, 2011 at 9:19 PM, Ingela Andin wrote: > >> Hi! >> >> Do you still get this using inets-5.6? In that case we must >> investigate this further? >> In such case do you have a minimal example that will reproduce the error? >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> 2011/6/6 Suma Shivaprasad : >> > Our app makes a lot of HTTP requests and we are facing this issue with >> both >> > inets-5.5.1 and 5.3.2. >> > >> > Basically our receive clause for the response is trying to match the >> request >> > id returned in httpc:request call >> > and we see that the match always fails with the wrong Request Id . >> > >> > We gave seen this mismatch in all 3 receive clauses for >> > stream_start, stream and stream_end >> > >> > What we observed after a lot of trial and error is that if the same pid >> > makes the http requests , the responses get kind of muddled up but if >> we >> > spawn a separate process for the httpc:request , it is better . But >> here >> > too we have seen some occurrences of this faulty behaviour. >> > >> > Has anyone faced this issue ? >> > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jul 14 03:29:31 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 13 Jul 2011 18:29:31 -0700 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: Why do you say that HTTP long-poll is bad for battery, but persistent TCP connections are not? A proper long-poll with a long timeout should be no different from a persistent TCP connection with the same time-out. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 12, 2011 at 1:25 PM, Matthew Hillsborough < matthew.hillsborough@REDACTED> wrote: > Greetings Erlang community, > > Let me further elaborate on my question that's in the subject of this > message. I tried to reach out with this question on StackOverflow, however I > did not have much luck there. Perhaps the community here can provide some > feedback here for me to let me know if I'm on the right track or if Erlang > is not the right tool for what I'm trying to accomplish. > > I'm building native mobile applications in both iOS and Android. These apps > require "realtime" updates from and to the server, same as many (but not > all) network-based application does (Facebook, social games like Words with > Friends, Finance applications, etc). The communication here is > bi-directional, in the sense that the server might have updates for the > mobile clients and the clients will be pushing data down to the server > whenever necessary. > > I think using HTTP long polling for this is over kill in the sense that > long polling can be detrimental to battery life, especially with a lot of > TCP setup/teardown for every HTTP connection the device needs to send out > through the wire. It might make sense to have the mobile applications use > persistent TCP sockets to establish a connection to the server, and send RPC > style commands to the server for all web service communication. This > ofcourse, would require a server to handle the long-lived TCP connection and > be able to speak to a web service once it makes sense of the data passed > down the TCP pipe. I'm thinking of passing data in plain text using JSON or > XML and then using some kind of Erlang interface to HTTP to call a web > service to handle all the REST type communication. The responses would then > go back to the "RPC" Erlang instance, which would send the updates to the > appropriate client(s). > > Perhaps an Erlang based RPC server would do well for a network based > application like this. It would allow for the mobile apps to send and > receive data from the server all over one connection without multiple > setup/tear down that individual HTTP requests would do. Since no web browser > is involved, we do not need to deal with the nuances of HTTP long-polling at > the mobile client level. I also haven't seen great long polling/keep-alive > support on the client-side in iOS, but that's irrelevant for the community > here. > > A lot of these "COMET" and long-polling/streaming servers are built with > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > better catered for the type of app I'm building, will make the client more > responsive, allow for receiving of updates from the server without > constantly polling the server, etc. > > I also looked into HTTP pipelining, but it doesn't look to be worth the > trouble when it comes to implementing it on the clients. Also, I'm not sure > if it would allow for bi-directional communication in the client<->server > communication channel. > > Am I completely out of line in thinking that building a custom solution in > Erlang is a good idea here? To my understanding, Erlang excels at servers > like this, and if I run the server on tcp/80, I should be able to avoid most > firewall/port issues. The client would need work to deal with timeouts, re > connections, acknowledging receipt of asynchronous requests, but that's not > Erlang's problem. > > Has anyone built something similar before? Should I just stick to a web > server and deal with "COMET" type technologies? (WebSockets, long-polling, > client-side polling). > > Was hoping someone could solidify that I'm not entirely insane for wanting > a better solution than HTTP would serve in this case, at least at the client > level. I'll still be using HTTP/REST extensively, the Erlang server would > just handle the persistent connections and messaging to the Web Service > (which would probably be something like Django or Rails). > > Sorry for the long post; I am just excited to get into the heads of people > who are smarter than I. > > Happy hacking! > > Matthew > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Thu Jul 14 08:37:43 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 14 Jul 2011 07:37:43 +0100 Subject: [erlang-questions] mnesia lookup weirdness In-Reply-To: References: Message-ID: For the record, I've found out what was causing this. A process was regularly executing ets:safe_fixtable(true) on this table, and not finishing off with ets:safe_fixtable(false). This meant that the behaviour of mnesia:dirty_first was affected, and was also causing increased memory consumption. Thanks to H?kan and Sverker for helping me find the root cause. cheers Chandru On 1 July 2011 14:44, Chandru wrote: > Hi, > > One of my production nodes running R12B-5 is exhibiting some strange > behaviour. > > mnesia:dirty_read(gtp_tid, mnesia:dirty_first(gtp_tid)). > > gives me an empty list. The table has 39K entries. How do I find out what > is happening? I tried using ets operations directly and get the same result > (which is what mnesia is doing behind the scenes anyway). I get the same > empty result using a transaction to read. Any clues? > > 96> ets:info(gtp_tid). > [{memory,213857900}, > {owner,<0.741.0>}, > {name,gtp_tid}, > {size,39088}, > {node,node@REDACTED}, > {named_table,true}, > {type,set}, > {keypos,2}, > {protection,public}] > > The same operation works ok on other tables. > > cheers > Chandru > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Thu Jul 14 10:20:43 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Thu, 14 Jul 2011 10:20:43 +0200 Subject: [erlang-questions] remsh In-Reply-To: References: <4E1C7106.6080105@ubiquity.it> Message-ID: <4E1EA6DB.1040709@ubiquity.it> Thank you for your feedback. That's exactly what I did eventually. :) Cheers, Carlo On 14/07/2011 03:23, Jon Watte wrote: > If I understand it correctly, "remsh" specifies to create an interactive > shell conneted to the remote node, but a command-line argument is > evaluated using the local node. > Try evaluating it as a rpc:call expression instead? You won't need a > -remsh at all then, instead passing the node argument to the rpc call. > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. > Nevertheless, whether we get there willingly or not, we shall soon have > lower consumption rates, because our present rates are unsustainable. > From sverker@REDACTED Thu Jul 14 11:36:12 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 14 Jul 2011 11:36:12 +0200 Subject: [erlang-questions] mnesia lookup weirdness In-Reply-To: References: Message-ID: <4E1EB88C.9010508@erix.ericsson.se> I have to confess... ets:first is not supposed to behave like that, returning already deleted keys, even if the table is fixated. You can of course have a race with a deleting process, but the key should have existed in the table some time during the execution of ets:first(). I think you suffer from this bug, fixed in R13B01: http://www.erlang.org/download/otp_src_R13B01.readme: OTP-8040 Various fixes in ETS: ets:first could return a deleted key in a fixated table. ets:lookup could return objects out of order if a deleted object was re-inserted into a fixed bag. ets:delete_object could fail to delete duplicate objects in a duplicate_bag. The irony being that you probably wouldn't had found the bug of the leaking ets:safe_fixtable now if ets:first had behaved like it should. Leaking ets:safe_fixtable can be a nasty thing as it might result in gradual performance degradation, both memory and cpu wise. Deleted objects are not deallocated and the table is not allowed to adjust its size (rehash) until it is "unfixated". /Sverker, Erlang/OTP Chandru wrote: > For the record, I've found out what was causing this. > > A process was regularly executing ets:safe_fixtable(true) on this table, and > not finishing off with ets:safe_fixtable(false). > > This meant that the behaviour of mnesia:dirty_first was affected, and was > also causing increased memory consumption. > > Thanks to H?kan and Sverker for helping me find the root cause. > > cheers > Chandru > > On 1 July 2011 14:44, Chandru wrote: > > >> Hi, >> >> One of my production nodes running R12B-5 is exhibiting some strange >> behaviour. >> >> mnesia:dirty_read(gtp_tid, mnesia:dirty_first(gtp_tid)). >> >> gives me an empty list. The table has 39K entries. How do I find out what >> is happening? I tried using ets operations directly and get the same result >> (which is what mnesia is doing behind the scenes anyway). I get the same >> empty result using a transaction to read. Any clues? >> >> 96> ets:info(gtp_tid). >> [{memory,213857900}, >> {owner,<0.741.0>}, >> {name,gtp_tid}, >> {size,39088}, >> {node,node@REDACTED}, >> {named_table,true}, >> {type,set}, >> {keypos,2}, >> {protection,public}] >> >> The same operation works ok on other tables. >> >> cheers >> Chandru >> >> >> > > From chandrashekhar.mullaparthi@REDACTED Thu Jul 14 11:41:41 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 14 Jul 2011 10:41:41 +0100 Subject: [erlang-questions] mnesia lookup weirdness In-Reply-To: <4E1EB88C.9010508@erix.ericsson.se> References: <4E1EB88C.9010508@erix.ericsson.se> Message-ID: On 14 July 2011 10:36, Sverker Eriksson wrote: > I have to confess... > > ets:first is not supposed to behave like that, returning already deleted > keys, > even if the table is fixated. You can of course have a race with a deleting > process, > but the key should have existed in the table some time during the execution > of ets:first(). > > I think you suffer from this bug, fixed in R13B01: > > http://www.erlang.org/**download/otp_src_R13B01.readme > **: > > OTP-8040 Various fixes in ETS: ets:first could return a deleted key in > a fixated table. ets:lookup could return objects out of order > if a deleted object was re-inserted into a fixed bag. > ets:delete_object could fail to delete duplicate objects in a > duplicate_bag. > > > The irony being that you probably wouldn't had found the bug of > the leaking ets:safe_fixtable now if ets:first had behaved like it should. > > Leaking ets:safe_fixtable can be a nasty thing as it might result in > gradual performance degradation, > both memory and cpu wise. Deleted objects are not deallocated and the table > is not allowed > to adjust its size (rehash) until it is "unfixated". > > Yes, you are right about this. The reason I stumbled upon this was because the erlang node was gradually eating up all available memory and crashing. I could find no obvious culprit, until I came across this! Chandru Chandru wrote: > For the record, I've found out what was causing this. > > A process was regularly executing ets:safe_fixtable(true) on this table, > and > not finishing off with ets:safe_fixtable(false). > > This meant that the behaviour of mnesia:dirty_first was affected, and was > also causing increased memory consumption. > > Thanks to H?kan and Sverker for helping me find the root cause. > > cheers > Chandru > > On 1 July 2011 14:44, Chandru > wrote: > > > >> Hi, >> >> One of my production nodes running R12B-5 is exhibiting some strange >> behaviour. >> >> mnesia:dirty_read(gtp_tid, mnesia:dirty_first(gtp_tid)). >> >> gives me an empty list. The table has 39K entries. How do I find out what >> is happening? I tried using ets operations directly and get the same >> result >> (which is what mnesia is doing behind the scenes anyway). I get the same >> empty result using a transaction to read. Any clues? >> >> 96> ets:info(gtp_tid). >> [{memory,213857900}, >> {owner,<0.741.0>}, >> {name,gtp_tid}, >> {size,39088}, >> {node,node@REDACTED}, >> {named_table,true}, >> {type,set}, >> {keypos,2}, >> {protection,public}] >> >> The same operation works ok on other tables. >> >> cheers >> Chandru >> >> >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jul 14 12:56:07 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 14 Jul 2011 12:56:07 +0200 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: References: Message-ID: <4E1ECB47.5070905@dev-extend.eu> On 07/13/2011 09:19 PM, Alex Arnon wrote: > Hi All, > > We're building a nontrivial Erlang application, which will provide a > potentially high-volume, public RESTful Web Service. As part of the API, > we will be providing both informative text (status, labels, layout > templates) and error messages. A twofold question, then, to the > experienced Grandmasters in the audience: > 1. How have you implemented localization of text? This includes > parameterized format strings. I don't know how well it works but there's always: http://www.trapexit.org/Gettext_-_An_internationalization_package. I'll be interested hearing what you find works best. > 2. Has anyone used composite error values? For example, instead of > {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, > LocalizedMessageId}, {message_args, [...]}... Should we steer clear of > these, and if so, what alternative would you suggest? OTP uses format_error functions to format the errors. You could use the same pattern in your case but internationalized. For example passing {error, badarg, What} to the function would give the formatted version in the currently selected language. -- Lo?c Hoguin Dev:Extend From alex.arnon@REDACTED Thu Jul 14 12:58:19 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Thu, 14 Jul 2011 13:58:19 +0300 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: <50fffd16-7e4d-4201-9c15-b44a9937a94c@v7g2000vbk.googlegroups.com> References: <50fffd16-7e4d-4201-9c15-b44a9937a94c@v7g2000vbk.googlegroups.com> Message-ID: On Thu, Jul 14, 2011 at 2:53 AM, Steve Davis wrote: > Given that "strings" are either ambiguous or implicitly constrained > inventions of data representation, then you may consider applying > something along the lines of: > > -record(text, {value, charset = utf8, locale = 'us-en'}). > > This is similar to what we had in mind, for localized parts of a more generic text structure (similar to iolists, but with records such as the above supported as well). Arguments might also be added as part of #text{}. This covers the "bit up to rendering" - everything would be rendered to UTF-8 binaries eventually for public consumption. ... snip ... BTW This problem isn't really an erlang/otp but one that every > platform that I have ever investigated. What makes erlang different is > that you can easily forsee a potential, and implementable, solution to > this legacy issue in CS. > > Yes, this is indeed a universal "problem". I have seen several solutions in other environments, usually using either a global string definitions file (macros/constants/hashes, or a simple flat data file) or a database (usually in larger systems). For our application we think a file containing all the definitions could be used, since the size of the message set will not be in the many-hundreds. Would you suggest that this be a properties/XML/whatever data file, or a module containing a translation function (or more)? -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Thu Jul 14 13:31:57 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Thu, 14 Jul 2011 14:31:57 +0300 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: <4E1ECB47.5070905@dev-extend.eu> References: <4E1ECB47.5070905@dev-extend.eu> Message-ID: On Thu, Jul 14, 2011 at 1:56 PM, Lo?c Hoguin wrote: > On 07/13/2011 09:19 PM, Alex Arnon wrote: > > Hi All, > > > > We're building a nontrivial Erlang application, which will provide a > > potentially high-volume, public RESTful Web Service. As part of the API, > > we will be providing both informative text (status, labels, layout > > templates) and error messages. A twofold question, then, to the > > experienced Grandmasters in the audience: > > 1. How have you implemented localization of text? This includes > > parameterized format strings. > > I don't know how well it works but there's always: > http://www.trapexit.org/Gettext_-_An_internationalization_package. > > I'll be interested hearing what you find works best. > Thank you, I shall look into it. And report :) > > > 2. Has anyone used composite error values? For example, instead of > > {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, > > LocalizedMessageId}, {message_args, [...]}... Should we steer clear of > > these, and if so, what alternative would you suggest? > > OTP uses format_error functions to format the errors. You could use the > same pattern in your case but internationalized. For example passing > {error, badarg, What} to the function would give the formatted version > in the currently selected language. > > -- > Lo?c Hoguin > Dev:Extend > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Jul 14 14:33:52 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 14 Jul 2011 14:33:52 +0200 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: References: Message-ID: <4E1EE230.9000007@gmail.com> On 07/13/2011 09:19 PM, Alex Arnon wrote: > We're building a nontrivial Erlang application, which will provide a > potentially high-volume, public RESTful Web Service. As part of the API, > we will be providing both informative text (status, labels, layout > templates) and error messages. A twofold question, then, to the > experienced Grandmasters in the audience: > 1. How have you implemented localization of text? This includes > parameterized format strings. At Klarna we are using (and maintaining) the gettext library, and it works very well. You'll find the latest version at https://github.com/etnt/gettext - the version at Jungerl is very much out of date. In particular, you'll probably want to use the new (not yet documented) macro ?STXT(...) instead of the ?TXT(...) macro. This is less error prone, since the fields and their meaning are more easily identified and do not have to be listed in order of occurrence. For example: ?STXT("Hello, $name$! Today's date is $date$ and the time is $time$.", [{date, current_date_as_string()}, {time, current_time_as_string(), {name, customer_name(CustomerID)}]) For the documentation (not very up to date, sorry), run 'make docs' and open doc/index.html in a browser. Tobbe also made some support for gettext in rebar: http://www.redhoterlang.com/entry/138dc1b1f8720f5c4f38864e4f0c338f To make it easier to actually get the translations into the system, or update translations as the original text changes, you may also want to use the Polish tool: http://www.redhoterlang.com/entry/8af2641026c0dfa27e54cb0da57eb392 http://www.youtube.com/watch?v=UdhE2YOkBCU http://www.redhoterlang.com/entry/98951c7196e0d91999bb238de5defe47 > 2. Has anyone used composite error values? For example, instead of > {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, > LocalizedMessageId}, {message_args, [...]}... Should we steer clear of > these, and if so, what alternative would you suggest? Who is expected to read these messages? End users, or developers? For developers, it's better if the message is in plain English. First, you may have people from different countries working on your code. Second, it's harder to google for the error message if the one you got was in Spanish, but the only people who have previously reported the problem got their messages in German. At the very least, always provide a summary or short identifier in English. /Richard From alex.arnon@REDACTED Thu Jul 14 16:46:50 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Thu, 14 Jul 2011 17:46:50 +0300 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: <4E1EE230.9000007@gmail.com> References: <4E1EE230.9000007@gmail.com> Message-ID: On Thu, Jul 14, 2011 at 3:33 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 07/13/2011 09:19 PM, Alex Arnon wrote: > >> We're building a nontrivial Erlang application, which will provide a >> potentially high-volume, public RESTful Web Service. As part of the API, >> we will be providing both informative text (status, labels, layout >> templates) and error messages. A twofold question, then, to the >> experienced Grandmasters in the audience: >> 1. How have you implemented localization of text? This includes >> parameterized format strings. >> > > At Klarna we are using (and maintaining) the gettext library, and it works > very well. You'll find the latest version at https://github.com/etnt/** > gettext - the version at Jungerl is very > much out of date. > > In particular, you'll probably want to use the new (not yet documented) > macro ?STXT(...) instead of the ?TXT(...) macro. This is less error prone, > since the fields and their meaning are more easily identified and do not > have to be listed in order of occurrence. For example: > > ?STXT("Hello, $name$! Today's date is $date$ and the time is $time$.", > [{date, current_date_as_string()}, > {time, current_time_as_string(), > {name, customer_name(CustomerID)}]) > > For the documentation (not very up to date, sorry), run 'make docs' and > open doc/index.html in a browser. > > Tobbe also made some support for gettext in rebar: > http://www.redhoterlang.com/**entry/**138dc1b1f8720f5c4f38864e4f0c33**8f > > To make it easier to actually get the translations into the system, or > update translations as the original text changes, you may also want to use > the Polish tool: > > http://www.redhoterlang.com/**entry/**8af2641026c0dfa27e54cb0da57eb3**92 > > http://www.youtube.com/watch?**v=UdhE2YOkBCU > > http://www.redhoterlang.com/**entry/**98951c7196e0d91999bb238de5defe**47 > > > Thank you, this looks promising! > > 2. Has anyone used composite error values? For example, instead of >> {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, >> LocalizedMessageId}, {message_args, [...]}... Should we steer clear of >> these, and if so, what alternative would you suggest? >> > > Who is expected to read these messages? End users, or developers? For > developers, it's better if the message is in plain English. First, you may > have people from different countries working on your code. Second, it's > harder to google for the error message if the one you got was in Spanish, > but the only people who have previously reported the problem got their > messages in German. At the very least, always provide a summary or short > identifier in English. > > /Richard > I was thinking of two purposes for the structured Reason: 1. The error handling code - which may receive extra information regarding the error itself. 2. Logging code - the handling code can forward the error to a logger, with attached message format + args etc. Localization of the message is probably pointless, I agree - I kind of overloaded [2] with [1]. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From chad@REDACTED Thu Jul 14 18:30:52 2011 From: chad@REDACTED (Chad DePue) Date: Thu, 14 Jul 2011 13:30:52 -0300 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: Message-ID: Matthew, Inaka has built native applications exactly like this for iPad and iPhone and started w/HTTP-comet style polling and we started with comet-style long polling but ended up at simple REST for the api commands and a TCP socket connection for the persistant stuff... The socket is far more efficient and quite simply with a mobile app there's no need to use long-polling. We also use Rails for admin but Erlang for all API commands, even HTTP. Ping me off-list and I can share some of what we learned. Chad DePue inakanetworks.com - development consulting | skype cdepue | @chaddepue +1 206.866.5707 On Tue, Jul 12, 2011 at 5:25 PM, Matthew Hillsborough < matthew.hillsborough@REDACTED> wrote: > Greetings Erlang community, > > Let me further elaborate on my question that's in the subject of this > message. I tried to reach out with this question on StackOverflow, however I > did not have much luck there. Perhaps the community here can provide some > feedback here for me to let me know if I'm on the right track or if Erlang > is not the right tool for what I'm trying to accomplish. > > I'm building native mobile applications in both iOS and Android. These apps > require "realtime" updates from and to the server, same as many (but not > all) network-based application does (Facebook, social games like Words with > Friends, Finance applications, etc). The communication here is > bi-directional, in the sense that the server might have updates for the > mobile clients and the clients will be pushing data down to the server > whenever necessary. > > I think using HTTP long polling for this is over kill in the sense that > long polling can be detrimental to battery life, especially with a lot of > TCP setup/teardown for every HTTP connection the device needs to send out > through the wire. It might make sense to have the mobile applications use > persistent TCP sockets to establish a connection to the server, and send RPC > style commands to the server for all web service communication. This > ofcourse, would require a server to handle the long-lived TCP connection and > be able to speak to a web service once it makes sense of the data passed > down the TCP pipe. I'm thinking of passing data in plain text using JSON or > XML and then using some kind of Erlang interface to HTTP to call a web > service to handle all the REST type communication. The responses would then > go back to the "RPC" Erlang instance, which would send the updates to the > appropriate client(s). > > Perhaps an Erlang based RPC server would do well for a network based > application like this. It would allow for the mobile apps to send and > receive data from the server all over one connection without multiple > setup/tear down that individual HTTP requests would do. Since no web browser > is involved, we do not need to deal with the nuances of HTTP long-polling at > the mobile client level. I also haven't seen great long polling/keep-alive > support on the client-side in iOS, but that's irrelevant for the community > here. > > A lot of these "COMET" and long-polling/streaming servers are built with > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is > better catered for the type of app I'm building, will make the client more > responsive, allow for receiving of updates from the server without > constantly polling the server, etc. > > I also looked into HTTP pipelining, but it doesn't look to be worth the > trouble when it comes to implementing it on the clients. Also, I'm not sure > if it would allow for bi-directional communication in the client<->server > communication channel. > > Am I completely out of line in thinking that building a custom solution in > Erlang is a good idea here? To my understanding, Erlang excels at servers > like this, and if I run the server on tcp/80, I should be able to avoid most > firewall/port issues. The client would need work to deal with timeouts, re > connections, acknowledging receipt of asynchronous requests, but that's not > Erlang's problem. > > Has anyone built something similar before? Should I just stick to a web > server and deal with "COMET" type technologies? (WebSockets, long-polling, > client-side polling). > > Was hoping someone could solidify that I'm not entirely insane for wanting > a better solution than HTTP would serve in this case, at least at the client > level. I'll still be using HTTP/REST extensively, the Erlang server would > just handle the persistent connections and messaging to the Web Service > (which would probably be something like Django or Rails). > > Sorry for the long post; I am just excited to get into the heads of people > who are smarter than I. > > Happy hacking! > > Matthew > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moxford@REDACTED Thu Jul 14 23:32:00 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 14 Jul 2011 14:32:00 -0700 Subject: [erlang-questions] Is Erlang a good tool for this particular project? In-Reply-To: References: <9486E9A0-9531-4A68-A32A-9EA4D72BCA9F@alum.mit.edu> Message-ID: On Wed, Jul 13, 2011 at 11:03 AM, Joel Meyer wrote: > On Wed, Jul 13, 2011 at 7:16 AM, Matthew Hillsborough > wrote: >> Thanks for posting links to this. Surprised Joe A. did not mention it in >> his post, it has him listed as an author. :) >> Just from a quick glimpse, this appears to be using MochiWeb, which is >> completely HTTP based. I am not entirely sure if that's the route I am > > Out of curiosity, is there a reason you wouldn't use thrift or protocol > buffers? Even if you don't use their RPC mechanisms, you could use one of > those as the serialized form for message passing (over plain sockets, if you > so desire). I think a binary serialization format will still be more compact > than JSON and faster to deserialize as well. Both of your points are true, but thrift and protobufs both have hard-IDLs. There are games you can play with non-IDL-based protocols (without giving up security) that you simply cannot do with binary protocols unless you force a client upgrade with the new IDL. -mox From hokan.stenholm@REDACTED Fri Jul 15 01:16:06 2011 From: hokan.stenholm@REDACTED (=?UTF-8?B?SMOla2FuIFN0ZW5ob2xt?=) Date: Fri, 15 Jul 2011 01:16:06 +0200 Subject: [erlang-questions] Best Practices: Localization of Erlang Applications. Also, Error Messages! In-Reply-To: References: <4E1EE230.9000007@gmail.com> Message-ID: <4E1F78B6.9060204@bredband.net> The thread "[erlang-questions] utf-8 PB" in the erlang mailing list (archive) during 2010-06-16/17/18 may be of interest to you. I (H?kan Stenholm) wrote a bunch of replies about using gettext and .po files in that thread. I used to work with internationalization/localization issues at Klarna back in 2005-2010, but not having worked there (or with erlang) since 2010-04 makes me fairly useless as a source for current information. On 2011-07-14 16.46, Alex Arnon wrote: > > > On Thu, Jul 14, 2011 at 3:33 PM, Richard Carlsson > > wrote: > > On 07/13/2011 09:19 PM, Alex Arnon wrote: > > We're building a nontrivial Erlang application, which will > provide a > potentially high-volume, public RESTful Web Service. As part > of the API, > we will be providing both informative text (status, labels, layout > templates) and error messages. A twofold question, then, to the > experienced Grandmasters in the audience: > 1. How have you implemented localization of text? This includes > parameterized format strings. > > > At Klarna we are using (and maintaining) the gettext library, and > it works very well. You'll find the latest version at > https://github.com/etnt/gettext - the version at Jungerl is very > much out of date. > > In particular, you'll probably want to use the new (not yet > documented) macro ?STXT(...) instead of the ?TXT(...) macro. This > is less error prone, since the fields and their meaning are more > easily identified and do not have to be listed in order of > occurrence. For example: > > ?STXT("Hello, $name$! Today's date is $date$ and the time is > $time$.", > [{date, current_date_as_string()}, > {time, current_time_as_string(), > {name, customer_name(CustomerID)}]) > > For the documentation (not very up to date, sorry), run 'make > docs' and open doc/index.html in a browser. > > Tobbe also made some support for gettext in rebar: > http://www.redhoterlang.com/entry/138dc1b1f8720f5c4f38864e4f0c338f > > To make it easier to actually get the translations into the > system, or update translations as the original text changes, you > may also want to use the Polish tool: > > http://www.redhoterlang.com/entry/8af2641026c0dfa27e54cb0da57eb392 > > http://www.youtube.com/watch?v=UdhE2YOkBCU > > http://www.redhoterlang.com/entry/98951c7196e0d91999bb238de5defe47 > > > > Thank you, this looks promising! > > > > 2. Has anyone used composite error values? For example, instead of > {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, > {message, > LocalizedMessageId}, {message_args, [...]}... Should we steer > clear of > these, and if so, what alternative would you suggest? > > > Who is expected to read these messages? End users, or developers? > For developers, it's better if the message is in plain English. > First, you may have people from different countries working on > your code. Second, it's harder to google for the error message if > the one you got was in Spanish, but the only people who have > previously reported the problem got their messages in German. At > the very least, always provide a summary or short identifier in > English. > > /Richard > > > I was thinking of two purposes for the structured Reason: > 1. The error handling code - which may receive extra information > regarding the error itself. > 2. Logging code - the handling code can forward the error to a logger, > with attached message format + args etc. > Localization of the message is probably pointless, I agree - I kind of > overloaded [2] with [1]. :) > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From coa@REDACTED Fri Jul 15 12:19:25 2011 From: coa@REDACTED (=?ISO-8859-1?Q?Cl=E1udio_Amaral?=) Date: Fri, 15 Jul 2011 11:19:25 +0100 Subject: [erlang-questions] compiler abstract syntax Message-ID: <4E20142D.2030801@dcc.fc.up.pt> Hello list! I have a generic question but I am looking for answers with specifics. I would like to do some of the Erlang's compilation steps and make some fun stuff with the abstract/intermediate representation and generate C code after. The problem is that it is hard to navigate in the compiler modules without some guidance knowledge and I am getting easily lost. Do you know where should I be really spending my time on? Where is the right place to play with the internal representation, what steps were made already, what steps are missing and where can I find the individual compilation/analysis steps. Sorry for not being more specific. Appreciate all the help I can get. Best, Cl?udio. From knut.nesheim@REDACTED Fri Jul 15 12:40:14 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Fri, 15 Jul 2011 12:40:14 +0200 Subject: [erlang-questions] Process scheduling and punishment Message-ID: Dear list, We have a case where a gen_server gets "slow" after it has handled many messages, while what it does stays exactly the same. We suspect the scheduling of the process changes. I was hoping someone on the list could shed some light on why this happens and if there is any way to avoid it. When repeatedly running the same test suite, after some time we notice random parts of the test suite getting two orders of magnitude slower. The tests query our server over HTTP and the roundtrip times goes from ~1ms to 75-100ms at a very sharp point. After this point, it stays at the same level until the gen_server is restarted. The CPU usage of the beam process stays around 5-10% and from etop we see no change. What happens is basically this: * From short-lived processes spawned by misultin we query a single gen_server while measuring wall clock between the point where we send the message and get the reply. From this point of view, the gen_server starts out fast, most calls take only a couple of hundred microseconds to complete. * Inside the gen_server we do very little work and from measuring the wallclock time we spend consistently less than 100 microseconds. * Around 10 times per second, from the gen_server we send a message containing roughly 1000 words to a logging process. * At the point where the misultin processes starts measuring the gen_server as slow, we still spend consistently less than 100 microseconds. * At this point, we also see messages(no more than one) in the message queue of the process, which is weird as end to end we are sequential so the process has nothing to do but handle these messages. * At no point do we see the logging process having messages in the queue. It is using the same amount of cpu in both states. Is it the case that our gen_server is "punished" due to overloading the logging process? Is there any way to measure if the VM considers our logging process to be overloaded? Is there any general form of "punishment" for very busy processes that might cause starvation for our gen_server? In our live system we have many of these gen_servers, but the request rate is much lower and they do very little logging(if at all). If it is the case that our gen_server is punished, what would happen when we have ten thousand of them? If all servers log at some point in it's life and one server goes crazy which causes the log process to be overloaded, will all servers be punished? Thanks Knut -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From dmercer@REDACTED Fri Jul 15 13:30:05 2011 From: dmercer@REDACTED (David Mercer) Date: Fri, 15 Jul 2011 06:30:05 -0500 Subject: [erlang-questions] Process scheduling and punishment In-Reply-To: References: Message-ID: <00f701cc42e2$8cb4c100$a61e4300$@com> On Friday, July 15, 2011, Knut Nesheim wrote: Is it possibly related to the > We have a case where a gen_server gets "slow" after it has handled > many messages, while what it does stays exactly the same. We suspect > the scheduling of the process changes. I was hoping someone on the > list could shed some light on why this happens and if there is any way > to avoid it. . . . > * Around 10 times per second, from the gen_server we send a message > containing roughly 1000 words to a logging process. . . . > * At no point do we see the logging process having messages in the > queue. It is using the same amount of cpu in both states. > > Is it the case that our gen_server is "punished" due to overloading > the logging process? Is there any way to measure if the VM considers > our logging process to be overloaded? Is there any general form of > "punishment" for very busy processes that might cause starvation for > our gen_server? > > In our live system we have many of these gen_servers, but the request > rate is much lower and they do very little logging(if at all). I'm guessing this is related to the cost of sending being proportional to the receiver's message queue. (Ref. last bullet point on http://www.erlang.org/documentation/doc-4.9.1/erts-4.9.1/notes.html.) You do say that you don't see messages accumulating in the logging process's queue, but I still have the feeling it is related to this. Cheers, DBM From jesper.louis.andersen@REDACTED Fri Jul 15 15:35:25 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 15 Jul 2011 15:35:25 +0200 Subject: [erlang-questions] Process scheduling and punishment In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 12:40, Knut Nesheim wrote: > Dear list, > > We have a case where a gen_server gets "slow" after it has handled > many messages, while what it does stays exactly the same. We suspect > the scheduling of the process changes. I was hoping someone on the > list could shed some light on why this happens and if there is any way > to avoid it. Just for the record... What version of OTP are we talking about here? -- J. From knut.nesheim@REDACTED Fri Jul 15 15:48:10 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Fri, 15 Jul 2011 15:48:10 +0200 Subject: [erlang-questions] Process scheduling and punishment In-Reply-To: References: Message-ID: On Fri, Jul 15, 2011 at 3:35 PM, Jesper Louis Andersen wrote: > Just for the record... What version of OTP are we talking about here? Sorry. The version is R14B03. Regards Knut -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From knut.nesheim@REDACTED Fri Jul 15 15:49:07 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Fri, 15 Jul 2011 15:49:07 +0200 Subject: [erlang-questions] Process scheduling and punishment In-Reply-To: <00f701cc42e2$8cb4c100$a61e4300$@com> References: <00f701cc42e2$8cb4c100$a61e4300$@com> Message-ID: On Fri, Jul 15, 2011 at 1:30 PM, David Mercer wrote: > I'm guessing this is related to the cost of sending being proportional to > the receiver's message queue. ?(Ref. last bullet point on > http://www.erlang.org/documentation/doc-4.9.1/erts-4.9.1/notes.html.) ?You > do say that you don't see messages accumulating in the logging process's > queue, but I still have the feeling it is related to this. > Thanks. This sounds like a possible explanation. Do you know if there is any way to measure/understand which process is slowed down? Regards Knut -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From mjtruog@REDACTED Fri Jul 15 19:46:15 2011 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 15 Jul 2011 10:46:15 -0700 Subject: [erlang-questions] Process scheduling and punishment In-Reply-To: References: Message-ID: <4E207CE7.7070103@gmail.com> One way that can happen is if lots of binaries are created faster than the garbage collector can easily consume them. Then memory consumption should be higher and the process should be slower. Generally, the way to deal with that, is handling all the binaries within a spawned (linked) process which is short-lived, to force (i.e., encourage) more immediate garbage collection. This code forces the garbage collection as much as possible https://github.com/okeuday/CloudI/blob/master/src/lib/unused/src/immediate_gc.erl , however, only testing would determine that such an extreme is necessary. - Michael On 07/15/2011 03:40 AM, Knut Nesheim wrote: > Dear list, > > We have a case where a gen_server gets "slow" after it has handled > many messages, while what it does stays exactly the same. We suspect > the scheduling of the process changes. I was hoping someone on the > list could shed some light on why this happens and if there is any way > to avoid it. > > When repeatedly running the same test suite, after some time we notice > random parts of the test suite getting two orders of magnitude slower. > The tests query our server over HTTP and the roundtrip times goes from > ~1ms to 75-100ms at a very sharp point. After this point, it stays at > the same level until the gen_server is restarted. The CPU usage of the > beam process stays around 5-10% and from etop we see no change. > > What happens is basically this: > * From short-lived processes spawned by misultin we query a single > gen_server while measuring wall clock between the point where we send > the message and get the reply. From this point of view, the gen_server > starts out fast, most calls take only a couple of hundred microseconds > to complete. > * Inside the gen_server we do very little work and from measuring the > wallclock time we spend consistently less than 100 microseconds. > * Around 10 times per second, from the gen_server we send a message > containing roughly 1000 words to a logging process. > * At the point where the misultin processes starts measuring the > gen_server as slow, we still spend consistently less than 100 > microseconds. > * At this point, we also see messages(no more than one) in the > message queue of the process, which is weird as end to end we are > sequential so the process has nothing to do but handle these messages. > * At no point do we see the logging process having messages in the > queue. It is using the same amount of cpu in both states. > > Is it the case that our gen_server is "punished" due to overloading > the logging process? Is there any way to measure if the VM considers > our logging process to be overloaded? Is there any general form of > "punishment" for very busy processes that might cause starvation for > our gen_server? > > In our live system we have many of these gen_servers, but the request > rate is much lower and they do very little logging(if at all). If it > is the case that our gen_server is punished, what would happen when we > have ten thousand of them? If all servers log at some point in it's > life and one server goes crazy which causes the log process to be > overloaded, will all servers be punished? > > Thanks > Knut From vincent.dephily@REDACTED Fri Jul 15 20:30:04 2011 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 15 Jul 2011 20:30:04 +0200 Subject: [erlang-questions] remsh In-Reply-To: References: <4E1C7106.6080105@ubiquity.it> Message-ID: <2109642.s8MKpIzpRt@moltowork> On Wednesday 13 July 2011 18:23:41 Jon Watte wrote: > If I understand it correctly, "remsh" specifies to create an interactive > shell conneted to the remote node, but a command-line argument is evaluated > using the local node. > Try evaluating it as a rpc:call expression instead? You won't need a -remsh > at all then, instead passing the node argument to the rpc call. That's how I understand it too, and it's really annoying that you can only use remsh interactively. rpc:call/4 is fine, but it only works one call at a time, needs to send all the results over the wire, and becomes useless as soon as you use terms that are tied to the VM. I've tried all maners of techniques to send input to a remsh: -eval, -noshell, -oldshell, -noinput, using a pipe, echoing in /proc/$PID/fd/0... The only thing that works is a hack with X that simulates keyboard input. Another thing that works is erl -noinput -name foobar@REDACTED \ -eval "try {ok,S,_} = erl_scan:string(\"$YOURCODEHERE\"), {ok,P} = erl_parse:parse_exprs(S), {value,V,_} = rpc:call('$NODE', erl_eval, exprs, [P,[]]), io:format(\"\\e[32mok\\e[m\n~p\n\", [V]) catch _:Err -> io:format(\"\\e[31merror\\e[m\n~p\n\", [Err]) end." \ -s erlang halt but it isn't much more elegant. So I've got some nontrivial code to run remotely and automatically, but for various reasons I do not want to have a module loaded on the remote node that does the work. Most languages with an interactive shell will happily read from standard input instead of from the tty, but /usr/bin/erl doesn't. It has -eval which is great, except that it doesn't work with a remote shell. Any idea ? -- Vincent de Phily From jwatte@REDACTED Fri Jul 15 20:55:29 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 15 Jul 2011 11:55:29 -0700 Subject: [erlang-questions] remsh In-Reply-To: <2109642.s8MKpIzpRt@moltowork> References: <4E1C7106.6080105@ubiquity.it> <2109642.s8MKpIzpRt@moltowork> Message-ID: Why wouldn't an rpc that does an eval() of erlang "shell" code input work? Seems easier than faking window-system-specific keycodes... Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Jul 15, 2011 at 11:30 AM, Vincent de Phily < vincent.dephily@REDACTED> wrote: > On Wednesday 13 July 2011 18:23:41 Jon Watte wrote: > > If I understand it correctly, "remsh" specifies to create an interactive > > shell conneted to the remote node, but a command-line argument is > evaluated > > using the local node. > > Try evaluating it as a rpc:call expression instead? You won't need a > -remsh > > at all then, instead passing the node argument to the rpc call. > > That's how I understand it too, and it's really annoying that you can only > use > remsh interactively. > > rpc:call/4 is fine, but it only works one call at a time, needs to send all > the results over the wire, and becomes useless as soon as you use terms > that > are tied to the VM. > > I've tried all maners of techniques to send input to a remsh: -eval, > -noshell, > -oldshell, -noinput, using a pipe, echoing in /proc/$PID/fd/0... The only > thing that works is a hack with X that simulates keyboard input. > > Another thing that works is > erl -noinput -name foobar@REDACTED \ > -eval "try {ok,S,_} = erl_scan:string(\"$YOURCODEHERE\"), > {ok,P} = erl_parse:parse_exprs(S), > {value,V,_} = rpc:call('$NODE', erl_eval, exprs, [P,[]]), > io:format(\"\\e[32mok\\e[m\n~p\n\", [V]) > catch > _:Err -> io:format(\"\\e[31merror\\e[m\n~p\n\", [Err]) > end." \ > -s erlang halt > but it isn't much more elegant. > > > So I've got some nontrivial code to run remotely and automatically, but for > various reasons I do not want to have a module loaded on the remote node > that > does the work. Most languages with an interactive shell will happily read > from > standard input instead of from the tty, but /usr/bin/erl doesn't. It has > -eval > which is great, except that it doesn't work with a remote shell. Any idea ? > -- > Vincent de Phily > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.dephily@REDACTED Sat Jul 16 14:21:59 2011 From: vincent.dephily@REDACTED (vincent dephily) Date: Sat, 16 Jul 2011 13:21:59 +0100 Subject: [erlang-questions] remsh In-Reply-To: References: <4E1C7106.6080105@ubiquity.it> <2109642.s8MKpIzpRt@moltowork> Message-ID: 2011/7/15, Jon Watte : > Why wouldn't an rpc that does an eval() of erlang "shell" code input work? > Seems easier than faking window-system-specific keycodes... I believe the code from my previous mail (requoted below) is as close as it gets to what you suggest. Unless I've missed a public API that is simpler than the scan/parse/eval dance I've used here, in which case I'd love some pointers. But I think that kind of solution is a missed opportunity for /usr/bin/erl anyway : we've got an amazing "-remsh" feature (which, if I understand correctly, is implemented partially at the VM level ?), but it is hampered by the impossibility of feeding code into it. I *can* get things done with rpc (plus eval if necessary), but that's much more complicated than it should be. >> Another thing that works is >> erl -noinput -name foobar@REDACTED \ >> -eval "try {ok,S,_} = erl_scan:string(\"$YOURCODEHERE\"), >> {ok,P} = erl_parse:parse_exprs(S), >> {value,V,_} = rpc:call('$NODE', erl_eval, exprs, [P,[]]), >> io:format(\"\\e[32mok\\e[m\n~p\n\", [V]) >> catch >> _:Err -> io:format(\"\\e[31merror\\e[m\n~p\n\", [Err]) >> end." \ >> -s erlang halt >> but it isn't much more elegant. -- Vincent de Phily From vincent.dephily@REDACTED Sat Jul 16 14:39:08 2011 From: vincent.dephily@REDACTED (vincent dephily) Date: Sat, 16 Jul 2011 13:39:08 +0100 Subject: [erlang-questions] remsh In-Reply-To: References: <4E1C7106.6080105@ubiquity.it> <2109642.s8MKpIzpRt@moltowork> Message-ID: 2011/7/16, vincent dephily : > But I think that kind of solution is a missed opportunity for > /usr/bin/erl anyway : we've got an amazing "-remsh" feature (which, if > I understand correctly, is implemented partially at the VM level ?), > but it is hampered by the impossibility of feeding code into it. I > *can* get things done with rpc (plus eval if necessary), but that's > much more complicated than it should be. Replying to myself after an eureka moment : /usr/bin/erl_call is what I was after. Dunno why I had forgotten about it. There used to be a showstopper bug with hostnames, but I believe that's fixed. Here's an easy way to run complex code in a remote node: echo "$YOURCODEHERE" | erl_call -name $REMOTENODE -e -- Vincent de Phily From carlsson.richard@REDACTED Sun Jul 17 12:48:27 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sun, 17 Jul 2011 12:48:27 +0200 Subject: [erlang-questions] compiler abstract syntax In-Reply-To: <4E20142D.2030801@dcc.fc.up.pt> References: <4E20142D.2030801@dcc.fc.up.pt> Message-ID: <4E22BDFB.1040806@gmail.com> On 2011-07-15 12:19, Cl?udio Amaral wrote: > I would like to do some of the Erlang's compilation steps and make some > fun stuff with the abstract/intermediate representation and generate C > code after. The problem is that it is hard to navigate in the compiler > modules without some guidance knowledge and I am getting easily lost. > > Do you know where should I be really spending my time on? Where is the > right place to play with the internal representation, what steps were > made already, what steps are missing and where can I find the individual > compilation/analysis steps. First of all, study the code in compile.erl. That will show you which the different phases of the compilation are (and the corresponding module for each phase), as well as some useful undocumented options like to_core, from_core, etc. And of course, read http://www.erlang.org/doc/man/compile.html#file-2 for info about the documented options, in particular the 'binary' option. The main compilation stages are: - preprocessing (macros, conditional compilation, includes) - source-level expansions, such as records to tuples - translation to Core Erlang + optimizations and optional inlining - translation to "Kernel Erlang" + pattern matching compilation and further optimizations - translation to Beam assembler + some more optimizations - encoding as Beam bytecode If you want to extract an intermediate representation and generate C code from it, you should either take the Core Erlang representation (which is fairly well documented and simpler to work with than the "abstract forms" that are used for source-level Erlang), or the Beam bytecode. /Richard From g9414002.pccu.edu.tw@REDACTED Sun Jul 17 19:51:34 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Mon, 18 Jul 2011 01:51:34 +0800 Subject: [erlang-questions] On Loop Message-ID: Hi, everyone. I did exercises on basic problems, with the following loop structure: for(I, Fp, Fi, Fc) -> case Fp(I) of true -> Fc(I), I1 = Fi(I), for(I1, Fp, Fi, Fc); false -> ok end. where I is loop index, Fp, Fi, and Fc are functions for loop predicate, incremental state, and loop context. Then I can write some normal for-loop instructions, such as 9x9 multiple table. (In my country, when we're children, we read and remember the 9x9 multiple table, so it's a famous material for us.) To write a program like "for(i=0; i<10; i++) { ... }" in C++, functions are defined as these: fp(I) -> I < 10. fi(I) -> I+1. It's for (i=2; i<10; i++) for (j=1; j<10; j++) cout << i << " * " << j << " = " << (i*j) << endl; so fc/1 shell be a loop. fc(I) -> for(I, fun fp/1, fun fi/1, fc1(I)). fc1(I) -> fun(J) -> io:format("~w x ~w = ~w~n", [I, J, I*J]). end. multiple() -> for(2, fun fp/1, fun fi/1, fun fc/1). Then, I got it work. Now I get an image that an index variable in imperative languages means a sequence of values or states, and then in functional languages it's corresponding to a list of values. So when I want 9x9 table, in C++ I need a variable but in Erlang I need a list with length 81. And, in my imagination, all data in Erlang is put in stack. So, when I use 81 cells, they occupy memory with 81 units, right? And, in my imagination, memory is free if a function execution ends, so by using loops above I partially save memory because of using nested for/4, right? -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Jul 17 20:28:59 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 17 Jul 2011 20:28:59 +0200 Subject: [erlang-questions] On Loop In-Reply-To: References: Message-ID: 2011/7/17 ??? (Yau-Hsien Huang) : > for(I, Fp, Fi, Fc) -> > ? ? case Fp(I) of > ? ? ? ? true -> > ? ? ? ? ? ? Fc(I), > ? ? ? ? ? ? I1 = Fi(I), > ? ? ? ? ? ? for(I1, Fp, Fi, Fc); > ? ? ? ? false -> > ? ? ? ? ? ? ok > ? ? end. I am not entirely sure what you are asking, but note that in your function, the recursive call to for/4 is in tail-position. This means that the tail-call-optimization is applicable. > And, in my imagination, all data in Erlang is put?in stack.?So, when > I use 81 cells, they occupy memory with 81?units, right? Not if the tail-call-optimization trigger! Then it is 1 unit, and the function uses as much memory as the C++ module a constant factor. And I dare say it does in your case. A more erlang-idiomatic way of encoding your problem is the following: [io:format("~B x ~B = ~B~n", [I, J, I*J]) || I <- lists:seq(2,10), J <- lists:seq(2,10)]. Here is the basic idea: 1. Create the list [2,3,...,9] = lists:seq(2,9) two times. 2. Use a List Comprehension. Iterate through the elements [2,3,...,9] and call it I. The semantics of list comprehensions are that for each of these, the list comprehension will carry out its second part which is to iterate J through [2,3,...,9]. In other words, we create the cartesian product of [2,...9] X [2,...,9]. 3. For each {I, J} pair, call io:format/2 on the result. It is succinct, but it does create the lists first though. So it does uses some more memory. -- J. From g9414002.pccu.edu.tw@REDACTED Mon Jul 18 10:24:39 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Mon, 18 Jul 2011 16:24:39 +0800 Subject: [erlang-questions] On Loop In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: ??? (Yau-Hsien Huang) Date: 2011/7/18 Subject: Re: [erlang-questions] On Loop To: Jesper Louis Andersen 2011/7/18 Jesper Louis Andersen > > And, in my imagination, all data in Erlang is put in stack. So, when > > I use 81 cells, they occupy memory with 81 units, right? > > Not if the tail-call-optimization trigger! Then it is 1 unit, and the > function uses as much memory as the C++ module a constant factor. And > I dare say it does in your case. > > A more erlang-idiomatic way of encoding your problem is the following: > > [io:format("~B x ~B = ~B~n", [I, J, I*J]) > || I <- lists:seq(2,10), J <- lists:seq(2,10)]. > > Here is the basic idea: > > 1. Create the list [2,3,...,9] = lists:seq(2,9) two times. > 2. Use a List Comprehension. Iterate through the elements [2,3,...,9] > and call it I. The semantics of list comprehensions are that for each > of these, the list comprehension will carry out its second part which > is to iterate J through [2,3,...,9]. In other words, we create the > cartesian product of [2,...9] X [2,...,9]. > 3. For each {I, J} pair, call io:format/2 on the result. > > It is succinct, but it does create the lists first though. So it does > uses some more memory. > > > -- > J. > Yes, and my question is: did I save memory by using nested loop? By using for/4, it may be a for(1, fun fp/1, fun fi/1, fun fc/1) where fp/1 is fp(I) -> I =< 81. When it's nested loop, main() -> for(1, fun fp/1, fun fi/1, fun fc/1). fp(I) -> I =< 9. ... fc(I) -> for (1, fun fp/1, fun fi/1, fc1(I)). fc1(I) -> fun(J) -> io:format("~w x ~w = ~w~n", [I, J, I*J]) end. Did I save memory when main/0 only run a function doing recursion 9 times, while in each time recursion it wakes up another function, fc/1, recurring 9 times? Following the consideration, can we say, "keep your Erlang program flat," that is, when executing there're not many too deep recursion and it's worth memory-restrict machines? -- Best Regards. --- Y-H. H. -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Mon Jul 18 15:54:34 2011 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Mon, 18 Jul 2011 15:54:34 +0200 Subject: [erlang-questions] ets and zombie process Message-ID: <4E243B1A.70807@ubiquity.it> Hello list, I have problem, as usual ;), this time with a process that refuses to die. This particular process is the owner of an ets table: ets:i(). id name type size mem owner sonde_in sonde_in ordered_set 1131 165217 <0.122.0> In the meanwhile. the owner process crashed, let's check: 3> whereis(probe_db_manager). undefined 4> probe_db_manager:sonde_in_last_probe(). ** exception exit: {noproc,{gen_server,call, [probe_db_manager,sonde_in_last_probe]}} in function gen_server:call/2 Ok, it is pushing daisies, but then why is the ets table still available? That made me suspicious, so I've checked running processes, and it's still there: 5> i(). ... <0.122.0> probe_db_manager:init/1 987 233278 0 gen_server:loop/6 9 ... So, to double check: 6> process_info(pid(0, 122,0)). [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<0.107.0>]}, {dictionary,[{'$ancestors',[starter_sonda,<0.47.0>]}, {'$initial_call',{probe_db_manager,init,1}}]}, {trap_exit,false}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.46.0>}, {total_heap_size,1974}, {heap_size,987}, {stack_size,9}, {reductions,233278}, {garbage_collection,[{min_bin_vheap_size,46368}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,845}]}, {suspending,[]}] 7> is_process_alive(pid(0, 122,0)). true 8> sys:get_status(pid(0, 122, 0)). {status,<0.122.0>, {module,gen_server}, [[{'$ancestors',[starter_sonda,<0.47.0>]}, {'$initial_call',{probe_db_manager,init,1}}], running,<0.107.0>,[], [{header,"Status for generic server probe_db_manager"}, {data,[{"Status",running}, {"Parent",<0.107.0>}, {"Logged events",[]}]}, {data,[{"State","1-VENERA-20110718150001"}]}]]} Parent is the supervisor. So, why the process <0.122.0> is in this sort of limbo? What do I have to do to make sure that after a crash it really goes away? Thanks, Carlo From tony@REDACTED Mon Jul 18 17:15:42 2011 From: tony@REDACTED (Tony Rogvall) Date: Mon, 18 Jul 2011 17:15:42 +0200 Subject: [erlang-questions] ets and zombie process In-Reply-To: <4E243B1A.70807@ubiquity.it> References: <4E243B1A.70807@ubiquity.it> Message-ID: On 18 jul 2011, at 15.54, Carlo Bertoldi wrote: > Hello list, > I have problem, as usual ;), this time with a process that refuses to die. > This particular process is the owner of an ets table: > ets:i(). > id name type size mem owner > sonde_in sonde_in ordered_set 1131 165217 <0.122.0> > > In the meanwhile. the owner process crashed, let's check: > 3> whereis(probe_db_manager). > undefined I would say that the server did not crash, it is just unregistered. Clearly the <0.122.0> is not registered as probe_db_manager so, either it was never registered or "someone" unregistered it. /Tony > 4> probe_db_manager:sonde_in_last_probe(). > ** exception exit: {noproc,{gen_server,call, > [probe_db_manager,sonde_in_last_probe]}} > in function gen_server:call/2 > > Ok, it is pushing daisies, but then why is the ets table still available? That made me suspicious, so I've checked running processes, and it's still there: > 5> i(). > ... > <0.122.0> probe_db_manager:init/1 987 233278 0 > gen_server:loop/6 9 > ... > > So, to double check: > 6> process_info(pid(0, 122,0)). > [{current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}, > {status,waiting}, > {message_queue_len,0}, > {messages,[]}, > {links,[<0.107.0>]}, > {dictionary,[{'$ancestors',[starter_sonda,<0.47.0>]}, > {'$initial_call',{probe_db_manager,init,1}}]}, > {trap_exit,false}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.46.0>}, > {total_heap_size,1974}, > {heap_size,987}, > {stack_size,9}, > {reductions,233278}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,845}]}, > {suspending,[]}] > 7> is_process_alive(pid(0, 122,0)). > true > 8> sys:get_status(pid(0, 122, 0)). > {status,<0.122.0>, > {module,gen_server}, > [[{'$ancestors',[starter_sonda,<0.47.0>]}, > {'$initial_call',{probe_db_manager,init,1}}], > running,<0.107.0>,[], > [{header,"Status for generic server probe_db_manager"}, > {data,[{"Status",running}, > {"Parent",<0.107.0>}, > {"Logged events",[]}]}, > {data,[{"State","1-VENERA-20110718150001"}]}]]} > > Parent is the supervisor. > > So, why the process <0.122.0> is in this sort of limbo? > What do I have to do to make sure that after a crash it really goes away? > > Thanks, > Carlo > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Have run Make so many times I dunno what's installed anymore" -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail@REDACTED Mon Jul 18 18:22:43 2011 From: mail@REDACTED (Tim Fletcher) Date: Mon, 18 Jul 2011 09:22:43 -0700 (PDT) Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: > Logging out can be accomplished by simply sending an "unauthenticated" HTTP > status code and a new Authenticate header that tells the client it needs new > credentials. Most browsers will cache HTTP Auth credentials until the end of the browser session, so although it looks like you've logged out, you'll often be able to get straight back in again without having to re-enter a password. This usability issue is more significant a problem than the overhead of SSL, IMO. An alternative to storing session data on the server is to use signed/ encrypted cookies. Stefan Tilkov outlined a general approach for doing this, which you can use in any language: http://www.innoq.com/blog/st/2009/06/devoxx_08_rest_patterns_and_an.html (slide 44 of 71) - ask user for name and password if no cookie passed - authenticate user - create auth token as username + expiry date - hash(auth token + server secret) - return cookie as hash + auth_token - server validates with algorithm on in-memory data Hope that helps. Cheers, Tim From erlang@REDACTED Mon Jul 18 21:13:38 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 18 Jul 2011 21:13:38 +0200 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: I'm still following this thread with interest thanks for all the information. It seems like there are two phases: 1) initial authentication Here there is some kind of challenge/response interaction If this succeeds the server sends a session cookie to the client. 2) Per-connection authentication. If 1) has succeeded, the client sends the session cookie to the in each new request. The sever uses this as a key into a database if the database lookup matches the correct user then everything is ok This has to be done ever time a new socket is opened. My original question also asked about openId - can't I get google (or something) to authenticate the user then do some kind of hand-over to me? If this is the case then I wouldn't have to bother with the details of passwords myself Cheers /Joe On Mon, Jul 18, 2011 at 6:22 PM, Tim Fletcher wrote: >> Logging out can be accomplished by simply sending an "unauthenticated" HTTP >> status code and a new Authenticate header that tells the client it needs new >> credentials. > > Most browsers will cache HTTP Auth credentials until the end of the > browser session, so although it looks like you've logged out, you'll > often be able to get straight back in again without having to re-enter > a password. This usability issue is more significant a problem than > the overhead of SSL, IMO. > > An alternative to storing session data on the server is to use signed/ > encrypted cookies. Stefan Tilkov outlined a general approach for doing > this, which you can use in any language: > > ?http://www.innoq.com/blog/st/2009/06/devoxx_08_rest_patterns_and_an.html > (slide 44 of 71) > > ?- ask user for name and password if no cookie passed > ?- authenticate user > ?- create auth token as username + expiry date > ?- hash(auth token + server secret) > ?- return cookie as hash + auth_token > ?- server validates with algorithm on in-memory data > > Hope that helps. > > Cheers, > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From wmacgyver@REDACTED Mon Jul 18 21:24:27 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Mon, 18 Jul 2011 15:24:27 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: Yes, you can use either google or facebook as a login in place of your own system. You'd use oAuth for this. Basically it involves bouncing them to either a google or facebook login page, with a url to return to on success. when it returns back, you'll have a token. we use it for exporting data out of google. more info at http://code.google.com/apis/gdata/articles/oauth.html in that article, there is also a link to google's oauth playground for testing and see how it works. the facebook one is documented at http://developers.facebook.com/docs/authentication/ Twitter also provides very similar service too. On Mon, Jul 18, 2011 at 3:13 PM, Joe Armstrong wrote: > My original question also asked about openId - can't I get google (or something) > to authenticate the user then do some kind of hand-over to me? If this > is the case then I wouldn't have to bother with the details of passwords myself -- Omnem crede diem tibi diluxisse supremum. From john@REDACTED Mon Jul 18 21:24:40 2011 From: john@REDACTED (John Kemp) Date: Mon, 18 Jul 2011 15:24:40 -0400 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: <60F7943F-929B-4ABF-97BE-F23F54916BD5@jkemp.net> Joe, On Jul 18, 2011, at 3:13 PM, Joe Armstrong wrote: > I'm still following this thread with interest thanks for all the information. > > It seems like there are two phases: There are actually (at least) three phases I can think of: i) A person provides some claim (I have the email address me@REDACTED) which the authenticating entity may verify (by sending an email or phone SMS message, for example). This can involve the creation of a shared secret (password) known only to the authenticating entity and the claimant. ii) Presentation of an identifier linked to this initial claim (account name, email address and so on) and the shared secret associated with the account identifier (your phase 1) iii) Session-based authentication, achieved by means of a session identifier (rather than an account identifier and secret) and potentially session-limited secret - such authentication is of the variety "I claim that I am the same entity which authenticate to you 2 hours ago by means of the email address me@REDACTED" and usually accomplished concretely by a "token" (such as a web cookie). > > 1) initial authentication > Here there is some kind of challenge/response interaction > If this succeeds the server sends a session cookie to the client. > > 2) Per-connection authentication. > If 1) has succeeded, the client sends the session cookie to the > in each new request. The sever uses this as a key into a database > if the database lookup matches the correct user then everything is ok > This has to be done ever time a new socket is opened. > > My original question also asked about openId - can't I get google (or something) > to authenticate the user then do some kind of hand-over to me? Yes, if you implement Google's (or Facebook's... and so on) protocol for making identity assertions about their users. OpenID isn't as much used these days as Facebook Connect and Google's equivalent protocol. > If this > is the case then I wouldn't have to bother with the details of passwords myself If you wish to rely on others to perform the actual authentication, and simply take indirect assertions of a user's authenticated status, that's correct. If you want to "know about the user", you may not get much (enough?) information from Google, FB et al, without asking the user directly to provide it. Such protocols are usually only supported in the context of a web browser (not other kinds of apps) since the web redirect is a good way of establishing some reasonable correspondence between the user at one site and the user at another site ("the user at my site Y is the one redirected from site X" as long as site X and Y trust each other to do the right thing). Cheers, - John > > Cheers > > /Joe > > > > > > > > > On Mon, Jul 18, 2011 at 6:22 PM, Tim Fletcher wrote: >>> Logging out can be accomplished by simply sending an "unauthenticated" HTTP >>> status code and a new Authenticate header that tells the client it needs new >>> credentials. >> >> Most browsers will cache HTTP Auth credentials until the end of the >> browser session, so although it looks like you've logged out, you'll >> often be able to get straight back in again without having to re-enter >> a password. This usability issue is more significant a problem than >> the overhead of SSL, IMO. >> >> An alternative to storing session data on the server is to use signed/ >> encrypted cookies. Stefan Tilkov outlined a general approach for doing >> this, which you can use in any language: >> >> http://www.innoq.com/blog/st/2009/06/devoxx_08_rest_patterns_and_an.html >> (slide 44 of 71) >> >> - ask user for name and password if no cookie passed >> - authenticate user >> - create auth token as username + expiry date >> - hash(auth token + server secret) >> - return cookie as hash + auth_token >> - server validates with algorithm on in-memory data >> >> Hope that helps. >> >> Cheers, >> Tim >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Mon Jul 18 23:39:56 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 18 Jul 2011 23:39:56 +0200 Subject: [erlang-questions] eunit update Message-ID: <4E24A82C.80205@gmail.com> Finally, I've managed to synch the OTP version of EUnit with my own development repository. They started drifting apart while I hadn't switched my own repo to Git yet. git fetch git@REDACTED:richcarl/otp.git eunit-2.2.0 Until OTP decides on what to do with their licensing, that version will remain as LGPL. The stand-alone version of EUnit is now dual licensed under Apache 2.0 + LGPL, and can be found here: https://github.com/richcarl/eunit /Richard From tristan.sloughter@REDACTED Tue Jul 19 03:27:58 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 18 Jul 2011 20:27:58 -0500 Subject: [erlang-questions] Static files served through Webmachine Message-ID: The Webmachine list seems pretty low traffic, so I thought I send this here to see if anyone had some interesting ideas. I've been mulling around in my head serving up static content in webapps I write with Webmachine. The main pattern I'm reaching is heavily using frontend tools like JQuery templates, Knockout.js, Spine, Backbone, etc and having the Webmachine backend essentially serve only as a RESTful JSON interface. No templating on the backend and things like that. Webmachine fits that part perfectly! It has made web development for me more than bearable but enjoyable for once. And I hope to release a sort of Erlang Webframework (a set of applications) in case others also find this method to be fitting to them as well. But the problem lies in the inefficient serving of static files like html, js and css. The call comes in, matches the last dispatch rule I have of '*' and goes to a resource to server static content like this: maybe_fetch_object(Ctx, Path) -> % if returns {true, NewCtx} then NewCtx has response_body case Ctx#ctx.response_body of undefined -> case file_exists(Ctx, Path) of {true, FullPath} -> {ok, Value} = file:read_file(FullPath), {true, Ctx#ctx{response_body=Value}}; false -> {false, Ctx} end; _Body -> {true, Ctx} end. An obvious optimization here is to cache files in memory and check that before doing a read_file. But a real solution to me seems to be not using Webmachine for static content at all somehow. The problem for an example url: /user/new I server up the file under /priv/user/new.html for /user/user_id I serve up /priv/user/show.html (show.html then references javascript that finds the user_id from the url and populates the content, so no server-side templating is needed. So the problem there is it relies on Webmachine dispatch rules and resource logic to know [user, new] -> /priv/user/new.html That means my two thoughts of nginx match on subdomain and sending like api.domain.com to Webmachine and anything else it would serve itself. Can anyone think of a way I can keep the nice URLs and serve the static html files through nginx or another webserver. Or am I just wasting time? :) Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Tue Jul 19 04:39:35 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 18 Jul 2011 20:39:35 -0600 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: > Can anyone think of a way I can keep the nice URLs and serve the static html > files through nginx or another webserver. The typical solution to this is to reverse proxy webmachine under nginx. You can set pattern matches in Nginx which will forward dynamic traffic to webmachine running on a different port (invisibly to the client) and serve static content in all its optimized nginx glory. Here's something from my own setup which forward /reset and /forgot to webmachine: location /forgot { proxy_pass http://127.0.0.1:8191 } location /reset { proxy_pass http://127.0.0.1:8191; } This will cause /reset to go to http://127.0.0.1:8191/reset. Webmachine runs on port 8191. jack. From tristan.sloughter@REDACTED Tue Jul 19 04:53:51 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 18 Jul 2011 21:53:51 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Yeah, I do things like that already with an nginx proxy in front of Webmachine. But my problem is wanting to be able have nice URLs like /user/user_id that if its a request for text/html returns a file from /priv/user/show.html while if its a request for application/json it returns the json representation of the user. Obviously I do that simply in Webmachine. But I'm not even sure if thats possible to filter out and proxy with nginx this way or f it is just a stupid thing to do or not. Thanks, Tristan On Mon, Jul 18, 2011 at 9:39 PM, Jack Moffitt wrote: > > Can anyone think of a way I can keep the nice URLs and serve the static > html > > files through nginx or another webserver. > > The typical solution to this is to reverse proxy webmachine under > nginx. You can set pattern matches in Nginx which will forward dynamic > traffic to webmachine running on a different port (invisibly to the > client) and serve static content in all its optimized nginx glory. > > Here's something from my own setup which forward /reset and /forgot to > webmachine: > > location /forgot { > proxy_pass http://127.0.0.1:8191 > } > > location /reset { > proxy_pass http://127.0.0.1:8191; > } > > This will cause /reset to go to http://127.0.0.1:8191/reset. > Webmachine runs on port 8191. > > jack. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Tue Jul 19 05:06:56 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 18 Jul 2011 21:06:56 -0600 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: > But my problem is wanting to be able have nice URLs like /user/user_id that > if its a request for text/html returns a file from /priv/user/show.html > while if its a request for application/json it returns the json Ah, I see the issue. I assume show.html is mostly empty and loads data from the json file? I'm not sure if nginx's matchers are sophisticated enough to route by content requested. You could return a 302 from webmachine, but this is probably going to make it less efficient than just dumping the file from memory. If you're willing to use the Rails-type .json/.html format stuff on the ends of your URLs, this becomes easy. Now you've made me curious whether web caching servers are smart enough to distinguish by content type as well. jack. From tristan.sloughter@REDACTED Tue Jul 19 05:24:20 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 18 Jul 2011 22:24:20 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Yeah, I agree a 302 would probably be less efficient. I'm not familiar with Rails doing .json/.html. My understand was Rails used urls like I was wanting to use: domain.com/user/ -- no .html. Do you mean something else where they use .json/.html? Good question about caching servers as well. On Mon, Jul 18, 2011 at 10:06 PM, Jack Moffitt wrote: > > But my problem is wanting to be able have nice URLs like /user/user_id > that > > if its a request for text/html returns a file from /priv/user/show.html > > while if its a request for application/json it returns the json > > Ah, I see the issue. I assume show.html is mostly empty and loads data > from the json file? I'm not sure if nginx's matchers are sophisticated > enough to route by content requested. You could return a 302 from > webmachine, but this is probably going to make it less efficient than > just dumping the file from memory. > > If you're willing to use the Rails-type .json/.html format stuff on > the ends of your URLs, this becomes easy. > > Now you've made me curious whether web caching servers are smart > enough to distinguish by content type as well. > > jack. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Tue Jul 19 05:49:35 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 18 Jul 2011 21:49:35 -0600 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: > I'm not familiar with Rails doing .json/.html. My understand was Rails used > urls like I was wanting to use: domain.com/user/ -- no .html. Do > you mean something else where they use .json/.html? Rails will parse the extension as the format, and defaults to HTML I believe. So calling /users/1.xml for example would do an XML dump of the dump. Similarly, you can do this for .json as well (but by default I think it does html and xml). This is redundant information due to this information being in HTTP headers preferably, but I think they did it as a practical solution for XMLHttpRequest, which has a limited ability to set headers. But if you did add such a thing, then nginx can easily proxy one and not the other or vice versa. jack. From sam@REDACTED Tue Jul 19 09:43:20 2011 From: sam@REDACTED (Sam Elliott) Date: Tue, 19 Jul 2011 08:43:20 +0100 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Rails does check the Accept: header as well, so quite a lot more processing goes into choosing the correct content-type for a request. On the XMLHttpRequest (xhr) instance, you can use xhr.setRequestHeader("Accept", ) and you can get to the xhr object via jQuery's beforeSend function to set this. Have you looked at using sendfile? This is essentially a way of letting erlang choose which file to send, but getting nginx to actually send it. You'll need to be proxying through either nginx or apache, but apart from that it's quite simple. Here's a rundown of how it works in nginx: http://wiki.nginx.org/XSendfile Sam On Tue, Jul 19, 2011 at 4:49 AM, Jack Moffitt wrote: > > > I'm not familiar with Rails doing .json/.html. My understand was Rails used > > urls like I was wanting to use: domain.com/user/ -- no .html. Do > > you mean something else where they use .json/.html? > > Rails will parse the extension as the format, and defaults to HTML I > believe. ?So calling /users/1.xml for example would do an XML dump of > the dump. Similarly, you can do this for .json as well (but by default > I think it does html and xml). > > This is redundant information due to this information being in HTTP > headers preferably, but I think they did it as a practical solution > for XMLHttpRequest, which has a limited ability to set headers. > > But if you did add such a thing, then nginx can easily proxy one and > not the other or vice versa. > > jack. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Tue Jul 19 10:10:44 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 19 Jul 2011 10:10:44 +0200 Subject: [erlang-questions] tracing tools Message-ID: Hi all, Tracing a system is the only reasonable way to really see what is going on with it for debugging or instructional purposes (for systems of at least "normal" complexity and size). If we could trace everything, then given appropriate tools we could see exactly what happened when at a post-mortem, without guessing from incomplete logs. Unfortunately, tracing everything is prohibitive in the really useful cases (i.e. with large systems). Tools like inviso have been created to try to alleviate this, but it doesn't feel like they succeeded (inviso will become deprecated). What I wonder is if there has been any work on improving tracing performance or if there are any plans in that direction. It might be so that the latest SMP improvements could allow for tracing to have less impact on the system's performance. Also, given that there are several tools that are using tracing in the background (like percept), I had moments where I wished that I could run those together with my "regular" debug-tracing. Since tracing is a global service, that would not be a good idea at the moment, but would it be possible to make it so? I am also considering some tools that will monitor a system and start/stop relevant tracing according to its configuration and that can "replay" a trace and allow investigating the system's state. best regards, Vlad From vladdu55@REDACTED Tue Jul 19 10:23:29 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 19 Jul 2011 10:23:29 +0200 Subject: [erlang-questions] [ANN] erlide updates Message-ID: [Sorry if you get duplicate notifications] Hi! It is the time of the year when we usually update the requirements for the erlide environment [*]. This time we'd like to be able to use Eclipse 3.6 and Erlang R13B04 as minimal baseline (from 3.5 and R12B). If there is any erlide user that has to use the older versions, please let me know. If I don't hear any complaint, the next release (August 23) will be the last one supporting 3.5 and R12 and also the first one supporting only 3.6+ and R13B+. It would also be useful to know if there is any user that is forced to use Java 1.5 instead of something newer (no answer = we can use something newer). Thank you. best regards, Vlad [*] For those who don't know what that is, it's the Eclipse-based IDE for Erlang. http://erlide.org From fdmanana@REDACTED Tue Jul 19 11:21:35 2011 From: fdmanana@REDACTED (Filipe David Manana) Date: Tue, 19 Jul 2011 10:21:35 +0100 Subject: [erlang-questions] memory allocation in NIFs Message-ID: Is there any potential issue when using allocation functions (malloc, calloc, etc) in NIFs other than enif_alloc? (I think it was introduced in R14) thanks -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From watson.timothy@REDACTED Tue Jul 19 12:06:30 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 19 Jul 2011 11:06:30 +0100 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: On 18 July 2011 20:24, Wilson MacGyver wrote: > Yes, you can use either google or facebook as a login in place of your > own system. > You'd use oAuth for this. Basically it involves bouncing them to > either a google or > facebook login page, with a url to return to on success. > I seem to remember there being some Erlang oAuth tools/utilities on github. Don't know if they're up to much, but it might be worth a wee look. From steve@REDACTED Tue Jul 19 12:12:43 2011 From: steve@REDACTED (Steve Strong) Date: Tue, 19 Jul 2011 12:12:43 +0200 Subject: [erlang-questions] web authentication In-Reply-To: References: Message-ID: <6AD216A99E0E4340A83915E0DD62565D@srstrong.com> Take a look at https://github.com/tim/erlang-oauth , might be of use -- Steve Strong, Director, id3as twitter.com/srstrong On Tuesday, 19 July 2011 at 12:06, Tim Watson wrote: > On 18 July 2011 20:24, Wilson MacGyver wrote: > > Yes, you can use either google or facebook as a login in place of your > > own system. > > You'd use oAuth for this. Basically it involves bouncing them to > > either a google or > > facebook login page, with a url to return to on success. > > I seem to remember there being some Erlang oAuth tools/utilities on > github. Don't know if they're up to much, but it might be worth a wee > look. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jul 19 14:00:45 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 19 Jul 2011 14:00:45 +0200 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter wrote: > Can anyone think of a way I can keep the nice URLs and serve the static html > files through nginx or another webserver. Put a Varnish accelerator in front of your system (https://www.varnish-cache.org/). That way, it doesn't matter if your backend is slow at serving files as the accelerator will just cache static stuff for you. In addition, you avoid the trouble of going through another system as a proxy for static content. Also, the solution is quite modular. On the development system, you don't need more than a single system running Erlang. In my opinion, there is little reason not to plug into the whole industry there is where the main point is to make serving HTTP go faster. Trying to beat that with Erlang is probably possible, but I don't think it is beneficial. Varnish is really really hard to beat. It is built specifically for being insanely fast and it serves its data from a shared mmap()'ing, scales to multiple CPUs and is a big blob of nasty C code. I'd rather stand on the shoulders here than trying to mess with it myself. -- J. From tristan.sloughter@REDACTED Tue Jul 19 17:24:32 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 19 Jul 2011 10:24:32 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Sam, Jesper both these sound great, thanks! I'm also going to look into what Jack was saying about how Rails handles some stuff. But I'm leaning towards Jesper's idea with Varnish being the best... I'm one of those people who scoff at most benchmarks so not sure I'll bother to do one for this, but maybe, if someone here can A) suggest the best setup for it B) if it makes sense at all or would just be another worthless benchmark that really gives no information about reality. Thanks! Tristan On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter > wrote: > > > Can anyone think of a way I can keep the nice URLs and serve the static > html > > files through nginx or another webserver. > > Put a Varnish accelerator in front of your system > (https://www.varnish-cache.org/). That way, it doesn't matter if your > backend is slow at serving files as the accelerator will just cache > static stuff for you. In addition, you avoid the trouble of going > through another system as a proxy for static content. Also, the > solution is quite modular. On the development system, you don't need > more than a single system running Erlang. > > In my opinion, there is little reason not to plug into the whole > industry there is where the main point is to make serving HTTP go > faster. Trying to beat that with Erlang is probably possible, but I > don't think it is beneficial. Varnish is really really hard to beat. > It is built specifically for being insanely fast and it serves its > data from a shared mmap()'ing, scales to multiple CPUs and is a big > blob of nasty C code. I'd rather stand on the shoulders here than > trying to mess with it myself. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Jul 19 18:47:10 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 19 Jul 2011 10:47:10 -0600 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: If you're already using Nginx and just want control over URLs, you have access to the standard rewrite module: http://wiki.nginx.org/HttpRewriteModule This is not 30x redirection btw, unless of course you want that. On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter wrote: > Sam, Jesper both these sound great, thanks! I'm also going to look into what > Jack was saying about how Rails handles some stuff. > But I'm leaning towards Jesper's idea with Varnish being the best... I'm one > of those people who scoff at most benchmarks so not sure I'll bother to do > one for this, but maybe, if someone here can A)?suggest?the best setup for > it B) if it makes sense at all or would just be another worthless benchmark > that really gives no information about reality. > Thanks! > Tristan > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen > wrote: >> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >> wrote: >> >> > Can anyone think of a way I can keep the nice URLs and serve the static >> > html >> > files through nginx or another webserver. >> >> Put a Varnish accelerator in front of your system >> (https://www.varnish-cache.org/). That way, it doesn't matter if your >> backend is slow at serving files as the accelerator will just cache >> static stuff for you. In addition, you avoid the trouble of going >> through another system as a proxy for static content. Also, the >> solution is quite modular. On the development system, you don't need >> more than a single system running Erlang. >> >> In my opinion, there is little reason not to plug into the whole >> industry there is where the main point is to make serving HTTP go >> faster. Trying to beat that with Erlang is probably possible, but I >> don't think it is beneficial. Varnish is really really hard to beat. >> It is built specifically for being insanely fast and it serves its >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >> blob of nasty C code. I'd rather stand on the shoulders here than >> trying to mess with it myself. >> >> >> -- >> J. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From allen.kim@REDACTED Tue Jul 19 21:54:20 2011 From: allen.kim@REDACTED (Allen Kim) Date: Tue, 19 Jul 2011 14:54:20 -0500 Subject: [erlang-questions] Upgrade from R13B02 to R14B03 In-Reply-To: Message-ID: Did make all .appup files for all system applications. I don't see the previous errors any more. Then, installed erlang otp r14b03 to target machine and when I run release_handler:install_release/1 I see the following error. =ERROR REPORT==== 19-Jul-2011::14:15:36 === Loading of /home/tracking/projects/tracking/erlang/lib/inets-5.6/ebin/httpd_manager.beam failed: badfile =ERROR REPORT==== 19-Jul-2011::14:15:36 === beam/beam_load.c(1300): Error loading module httpd_manager: use of opcode 151; this emulator supports only up to 148 =ERROR REPORT==== 19-Jul-2011::14:15:36 === beam/beam_load.c(1300): Error loading module tftp_engine: use of opcode 151; this emulator supports only up to 148 =ERROR REPORT==== 19-Jul-2011::14:15:36 === Loading of /home/tracking/projects/tracking/erlang/lib/inets-5.6/ebin/tftp_engine.beam failed: badfile anyone? From: Allen Kim > Date: Thu, 14 Jul 2011 15:31:22 -0500 To: "erlang-questions@REDACTED" > Cc: Allen Kim > Subject: Upgrade from R13B02 to R14B03 Hi, I have installed elrang otp R14B03 on my machine. Then, I wanted to upgrade erlang otp from R13B02 to R14B03. I have myapp-1.9.0.rel like this: {release,{"tracking","1.9.0"},{erts,"5.7.3"}, [ {kernel,"2.13.3"}, {sasl,"2.1.7"}, {stdlib,"1.16.3"}, {compiler, "4.6.3", load}, {crypto, "1.6.1", permanent}, {edoc,"0.7.6.4", load}, {et,"1.3.2"}, {syntax_tools,"1.6.3", load}, {gs,"1.5.10"}, {inets,"5.1.3"}, {mnesia,"4.4.11", permanent}, {observer,"0.9.8.1"}, {os_mon,"2.2.3", permanent}, {otp_mibs,"1.0.5", load}, {snmp,"4.13.5", load}, {public_key,"0.3", load}, {ssl,"3.10.4", load}, {tools,"2.6.4", load}, {webtool,"0.8.4", load}, {xmerl,"1.2.1", load}, {myapp,"1.0.0", load} ]}. and, I have myapp-2.0.0.rel like this: {release,{"tracking","2.0.0"},{erts,"5.8.4"}, [ {kernel,"2.14.4"}, {sasl,"2.1.9.4"}, {stdlib,"1.17.4"}, {compiler, "4.7.4", load}, {crypto, "2.0.3", permanent}, {edoc,"0.7.8", load}, {et,"1.4.3"}, {syntax_tools,"1.6.7.1", load}, {gs,"1.5.13"}, {inets,"5.6"}, {mnesia,"4.4.19", permanent}, {observer,"0.9.9"}, {os_mon,"2.2.6", permanent}, {otp_mibs,"1.0.6", load}, {snmp,"4.20", load}, {public_key,"0.12", load}, {ssl,"4.1.5", load}, {tools,"2.6.6.4", load}, {webtool,"0.8.8", load}, {xmerl,"1.2.9", load}, {myapp,"1.0.0", load} ]}. when I run make_relup command, I see the following error. error = systools:make_relup("myapp-2.0.0", ["myapp-1.9.0"], ["myapp-1.9.0"], [restart_emulator, {path, ["/home/allen/releases/1.9.0/lib/*/ebin", "/home/allen/applications/*/ebin"]}]) No release upgrade script entry for kernel-2.14.4 to kernel-2.13.3 in file "/home/azoogle/projects/releases/1.9.0/lib/kernel-2.14.4/ebin/kernel.appup" that appup file looks like this. $ cat /home/azoogle/projects/releases/1.9.0/lib/kernel-2.14.4/ebin/kernel.appup {"2.14.4",[],[]}. Does it mean do I have to write all .appup files for all application that I want to upgrade with erlang version upgrade? Is there any one who upgraded erlang version with systools:make_relup? Thanks for your reply in advance Allen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Wed Jul 20 00:16:38 2011 From: james@REDACTED (James Aimonetti) Date: Tue, 19 Jul 2011 15:16:38 -0700 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list Message-ID: <4E260246.9090507@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Was wondering the reasoning for lists:all/2 returning true when passed an empty list as the second parameter? lists:all(some_fun/1, []) => 'true' The docs state "Returns true if Pred(Elem) returns true for all elements Elem in List, otherwise false."[1] The definition in lists.erl for all/2 shows why it returns true: all(Pred, [Hd|Tail]) -> case Pred(Hd) of true -> all(Pred, Tail); false -> false end; all(Pred, []) when is_function(Pred, 1) -> true. To me, at least, since Pred(Elem) never returned true, the outcome should be false. I might write it as: all(_, []) -> false; all(Pred, L) -> all_(Pred, L). all_(Pred, [Hd|Tail]) -> case Pred(Hd) of true -> all(Pred, Tail); false -> false end; all_(Pred, []) when is_function(Pred, 1) -> true. lists:any/2 returns false, as expected, for the empty list, but for similar reasons as lists:all/2. Anyway, minor quibble, but I was curious of the reasoning... James [1] http://erldocs.com/R14B/stdlib/lists.html?i=0&search=lists%20all#all/2 - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOJgJGAAoJENc77s1OYoGgvskIAI+B9jZfbTWrNarFraF68Enw Vpjt11jedrDe1nLPLXuVWKsV+lckqfET/EWl8f0GrrDOJlNXYucpw5V/iSQKyOfK KIDvQRFvgjzqky3gZb259HF9SAblwdPIQ7M2pXtN9jejOPmT1az/b6NyYt/adAM/ r4WTllf/OTFPyDhqQ27cKRgfRAv4nt4EmESnYTfJkduxSmgps7IW7XzGrvlCDTQs GboIwX7ICFcoFdOnxtlCN7iPQC2w5KYOvxcuF8RK788umY5DHZLrq3PqhkUjrtM+ A1ZUiefkmo3Z+IOTlYeYeLN9zpAmMVQ93gCd1vgdEfzDUHmcyN8XXcuyj8OF6KY= =QHzR -----END PGP SIGNATURE----- From james@REDACTED Wed Jul 20 00:34:19 2011 From: james@REDACTED (James Aimonetti) Date: Tue, 19 Jul 2011 15:34:19 -0700 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: References: <4E260246.9090507@2600hz.com> Message-ID: <4E26066B.8000507@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Yeah, documentation update would disambiguate the issue. Like I said, minor quibble... :) On 07/19/2011 03:29 PM, Alexander Krasnukhin wrote: > It's the standard way a filter works. Otherwise you will be forced to check > if list is empty every time you want to use this function. After writing > this check several times you will fix function back to return true for empty > lists. > > But I agree the documentation is unclear. What about "Returns false if > Pred(Elem) returns false for some element in List, otherwise true" > > On Wed, Jul 20, 2011 at 12:16 AM, James Aimonetti wrote: > > Was wondering the reasoning for lists:all/2 returning true when passed > an empty list as the second parameter? > > lists:all(some_fun/1, []) => 'true' > > The docs state "Returns true if Pred(Elem) returns true for all elements > Elem in List, otherwise false."[1] > > The definition in lists.erl for all/2 shows why it returns true: > > all(Pred, [Hd|Tail]) -> > case Pred(Hd) of > true -> all(Pred, Tail); > false -> false > end; > all(Pred, []) when is_function(Pred, 1) -> true. > > To me, at least, since Pred(Elem) never returned true, the outcome > should be false. I might write it as: > > all(_, []) -> false; > all(Pred, L) -> > all_(Pred, L). > > all_(Pred, [Hd|Tail]) -> > case Pred(Hd) of > true -> all(Pred, Tail); > false -> false > end; > all_(Pred, []) when is_function(Pred, 1) -> true. > > lists:any/2 returns false, as expected, for the empty list, but for > similar reasons as lists:all/2. > > Anyway, minor quibble, but I was curious of the reasoning... > > James > > [1] http://erldocs.com/R14B/stdlib/lists.html?i=0&search=lists%20all#all/2 _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions >> - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOJgZrAAoJENc77s1OYoGghrkIAK1BgvOz8n672yRmkjUoQSjI 0cz+/ioOXJ3m/hgUjzLKTVfPKh+QDMyoBZ1ylh5RB97cTuUOyMcQKh3NASF8hKsD Niw+jhy7i1LjeUMWffL4tE8E/jqcEl9Hiy+7iL3Ex8yeFFBmyAekrCMagjYVmQTF Z5lU7RWORhkT1Wm49+8mcbseN3kOJ5Fzz7GW55VoAfTna8hEHScsBxoKzvrarwsK Y3crQ5dWPV0J6GuufExZbuhTHyx6X3elPcs8P5lyAaIcsuaZ9ijjvYPq8UdSmfQE qkxCrAFAKduh5YDWxyvOGnDgay1ypW4b9eGSDncBbUXMd9xVK+jwccjjNYhaF74= =bwCn -----END PGP SIGNATURE----- From freza@REDACTED Wed Jul 20 00:43:24 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 19 Jul 2011 23:43:24 +0100 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <4E260246.9090507@2600hz.com> References: <4E260246.9090507@2600hz.com> Message-ID: <20110719224324.GA2057@hanele.lan> # James Aimonetti 2011-07-19: > Was wondering the reasoning for lists:all/2 returning true when passed > an empty list as the second parameter? See http://en.wikipedia.org/wiki/Universal_quantification#The_empty_set. Intuitively: If universal quantifier evaluates to false that would mean there was at least one element violating the predicate, right? Now how could that possibly happen in a set with no elements at all? BR, -- Jachym From james@REDACTED Wed Jul 20 01:28:31 2011 From: james@REDACTED (James Aimonetti) Date: Tue, 19 Jul 2011 16:28:31 -0700 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <20110719224324.GA2057@hanele.lan> References: <4E260246.9090507@2600hz.com> <20110719224324.GA2057@hanele.lan> Message-ID: <4E26131F.4030002@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bingo, winner. Thanks Jachym. Maths ftw! On 07/19/2011 03:43 PM, Jachym Holecek wrote: > # James Aimonetti 2011-07-19: >> Was wondering the reasoning for lists:all/2 returning true when passed >> an empty list as the second parameter? > > See http://en.wikipedia.org/wiki/Universal_quantification#The_empty_set. > > Intuitively: If universal quantifier evaluates to false that would mean > there was at least one element violating the predicate, right? Now how > could that possibly happen in a set with no elements at all? > > BR, > -- Jachym > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOJhMfAAoJENc77s1OYoGgDBIH/3fu9Io4ZT0CLRd2pShhUxVL VWAazvSuF5WYGKsRKuCXWiSOB34RKHvZqGT1gCqXHnVpS+tiwrDQDbtjVYQozXxK F9AGej0T65qsOTRf6qkNy1eheztRF4ZYOl94l0LhNUkQ1MMWtneXk7CVpk8cUyPl 1O9KVYt71DX/kAMNl60OeQ7XSXKifngRA7GS1ZfyvjiT06FyDrs/vPrSyDrA0ZT3 t9bMwrTSRj0vCKR+QpekVAXy2dt0LZLsuYq0MxS+VYGPsyeAlCTMiWuVxw83l8p9 74lMcOB9gIh50yymYk6ifuBXt8cs78BUYNfmkEpID29+9QZAwrNgyYPas25cuOg= =uC+s -----END PGP SIGNATURE----- From mjtruog@REDACTED Wed Jul 20 03:09:49 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 19 Jul 2011 18:09:49 -0700 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <4E26066B.8000507@2600hz.com> References: <4E260246.9090507@2600hz.com> <4E26066B.8000507@2600hz.com> Message-ID: <4E262ADD.6000608@gmail.com> I asked about this almost exactly 2 years ago (5 more days). However, no response came up on the mailing list. I think you might be expected to modify the documentation in github. The choice of true for lists:all/2 with an empty list still seems arbitrary to me. However, I don't think there is a good argument for the choice being wrong. Also, even if there was such an argument, it doesn't seem like the lists module would be changed because of the legacy code that depends on it. On 07/19/2011 03:34 PM, James Aimonetti wrote: > Yeah, documentation update would disambiguate the issue. Like I said, > minor quibble... :) > > On 07/19/2011 03:29 PM, Alexander Krasnukhin wrote: > > It's the standard way a filter works. Otherwise you will be forced to check > > if list is empty every time you want to use this function. After writing > > this check several times you will fix function back to return true for empty > > lists. > > > But I agree the documentation is unclear. What about "Returns false if > > Pred(Elem) returns false for some element in List, otherwise true" > > > On Wed, Jul 20, 2011 at 12:16 AM, James Aimonetti wrote: > > > Was wondering the reasoning for lists:all/2 returning true when passed > > an empty list as the second parameter? > > > lists:all(some_fun/1, []) => 'true' > > > The docs state "Returns true if Pred(Elem) returns true for all elements > > Elem in List, otherwise false."[1] > > > The definition in lists.erl for all/2 shows why it returns true: > > > all(Pred, [Hd|Tail]) -> > > case Pred(Hd) of > > true -> all(Pred, Tail); > > false -> false > > end; > > all(Pred, []) when is_function(Pred, 1) -> true. > > > To me, at least, since Pred(Elem) never returned true, the outcome > > should be false. I might write it as: > > > all(_, []) -> false; > > all(Pred, L) -> > > all_(Pred, L). > > > all_(Pred, [Hd|Tail]) -> > > case Pred(Hd) of > > true -> all(Pred, Tail); > > false -> false > > end; > > all_(Pred, []) when is_function(Pred, 1) -> true. > > > lists:any/2 returns false, as expected, for the empty list, but for > > similar reasons as lists:all/2. > > > Anyway, minor quibble, but I was curious of the reasoning... > > > James > > > [1] http://erldocs.com/R14B/stdlib/lists.html?i=0&search=lists%20all#all/2 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed Jul 20 04:44:11 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 20 Jul 2011 14:44:11 +1200 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <4E260246.9090507@2600hz.com> References: <4E260246.9090507@2600hz.com> Message-ID: <5A8F6EBA-4A78-44BE-A904-3FF372303B02@cs.otago.ac.nz> On 20/07/2011, at 10:16 AM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Was wondering the reasoning for lists:all/2 returning true when passed > an empty list as the second parameter? > > lists:all(some_fun/1, []) => 'true' Because that's the way universal quantifiers are *SUPPOSED* to work. lists:all(F, L) is true [assuming the types are right] precisely when there is no element X of L such that F(X) is false. Since [] has no elements, it cannot have any that would falsify some_fun. It's just the same in Lisp: (EVERY predicate list...) returns false as soon as any invocation of predicate returns false. If the end of a sequence is reached, every returns true. Thus, every returns true if and only if every invocation of predicate returns true. It's just the same in Smalltalk: aSequence allSatisfy: [:x | some test on x] answers false if aSequence has some element falsifying the test, true otherwise, so #() allSatisfy: anything is true. It's just the same in Haskell. It's also just the same in mathematics, where (?x??) p(x) is required to be true. > The definition in lists.erl for all/2 shows why it returns true: No, the definition shows HOW it returns true, but not WHY. WHY is because all/2 is an iterated "and", and the identity for "and" is "true". One of the major weaknesses in the Aristotelian syllogisms was the assumption that "all Xs are Ys" entailed there being at least one X. The convention used in Erlang is that of modern logic. In just the same way, existential quantifiers are supposed to return false if you quantify over an empty set, sums to return 0, products to return 1, and other such operations to return their identities. From ok@REDACTED Wed Jul 20 04:56:56 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 20 Jul 2011 14:56:56 +1200 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <4E262ADD.6000608@gmail.com> References: <4E260246.9090507@2600hz.com> <4E26066B.8000507@2600hz.com> <4E262ADD.6000608@gmail.com> Message-ID: On 20/07/2011, at 1:09 PM, Michael Truog wrote: > The choice of true for lists:all/2 with an empty list still seems arbitrary to me. It is anything but! Let's switch to Haskell for a minute. foldr f e [] = e foldr f e (x:xs) = f x (foldr f e xs) That is, foldr f e [x1,x2,...,xn] = f x1 (f x2 ... (f xn e) ...) Note that for uniformity, foldr f e [x] has to be f x e. and = foldr (&&) True or = foldr (||) False "and" tests whether all the elements of a list are true "or" tests whether at least one element of a list is true Note that *no* value other than True could be used for "and"; *no* value other than False could be used for "or". Try the opposite truth value and it just plain won't work. map f [] = [] map f (x:xs) = f x : map f xs all p = and . map p any p = or . map p That is, apply a predicate p to all the elements of a list, and then check whether all (or any) of the results are true. Expand out these definitions and you get all p [] = True all p (x:xs) = p x && all p xs any p [] = False any p (x:xs) = p x || any p xs Once again, plugging anything other than True into the base case of "all" could not possibly work. aristotelian_all p [] = False aristotelian_all p (x:xs) = all p (x:xs) or aristotelian_all p xs = not (null xs) && all p xs and however you slice it, the unmotivated requirement to return False for any empty list just makes the code longer and less comprehensible. You should see what horrors it inflicts on the code that USES it! > However, I don't think there is a good argument for the choice being wrong. You would have to convince every mathematician and logician on the planet that they were doing quantifiers wrong. The documentation probably didn't say anything about the issue because it doesn't make any more sense for all(P, []) to return false than for sum([]) to return -792. From overminddl1@REDACTED Wed Jul 20 05:48:37 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Tue, 19 Jul 2011 21:48:37 -0600 Subject: [erlang-questions] [ANN] erlide updates In-Reply-To: References: Message-ID: Newer is good for me. Erlide is great. On Jul 19, 2011 2:24 AM, "Vlad Dumitrescu" wrote: > [Sorry if you get duplicate notifications] > > Hi! > > It is the time of the year when we usually update the requirements for > the erlide environment [*]. This time we'd like to be able to use > Eclipse 3.6 and Erlang R13B04 as minimal baseline (from 3.5 and R12B). > > If there is any erlide user that has to use the older versions, please > let me know. If I don't hear any complaint, the next release (August > 23) will be the last one supporting 3.5 and R12 and also the first one > supporting only 3.6+ and R13B+. It would also be useful to know if > there is any user that is forced to use Java 1.5 instead of something > newer (no answer = we can use something newer). > > Thank you. > > best regards, > Vlad > > [*] For those who don't know what that is, it's the Eclipse-based IDE > for Erlang. http://erlide.org > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Wed Jul 20 06:30:02 2011 From: james@REDACTED (James Aimonetti) Date: Tue, 19 Jul 2011 21:30:02 -0700 Subject: [erlang-questions] lists:all/2 unexpected result for the empty list In-Reply-To: <5A8F6EBA-4A78-44BE-A904-3FF372303B02@cs.otago.ac.nz> References: <4E260246.9090507@2600hz.com> <5A8F6EBA-4A78-44BE-A904-3FF372303B02@cs.otago.ac.nz> Message-ID: <4E2659CA.1050503@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for the lesson. Both this reply and your other one in another part of the thread were illuminating. James On 07/19/2011 07:44 PM, Richard O'Keefe wrote: > > On 20/07/2011, at 10:16 AM, James Aimonetti wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Was wondering the reasoning for lists:all/2 returning true when passed >> an empty list as the second parameter? >> >> lists:all(some_fun/1, []) => 'true' > > Because that's the way universal quantifiers are *SUPPOSED* to work. > > lists:all(F, L) is true [assuming the types are right] > precisely when there is no element X of L such that F(X) is false. > > Since [] has no elements, it cannot have any that would falsify some_fun. > > It's just the same in Lisp: > (EVERY predicate list...) returns false as soon as any invocation of predicate > returns false. If the end of a sequence is reached, every returns true. > Thus, every returns true if and only if every invocation of predicate returns true. > > It's just the same in Smalltalk: > > aSequence allSatisfy: [:x | some test on x] > answers false if aSequence has some element falsifying the test, > true otherwise, so #() allSatisfy: anything is true. > > It's just the same in Haskell. > > It's also just the same in mathematics, where > (?x??) p(x) > is required to be true. > >> The definition in lists.erl for all/2 shows why it returns true: > > No, the definition shows HOW it returns true, but not WHY. > WHY is because all/2 is an iterated "and", and the identity > for "and" is "true". > > One of the major weaknesses in the Aristotelian syllogisms was > the assumption that "all Xs are Ys" entailed there being at > least one X. The convention used in Erlang is that of modern > logic. > > In just the same way, existential quantifiers are supposed to return > false if you quantify over an empty set, sums to return 0, > products to return 1, and other such operations to return their > identities. > > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOJlnJAAoJENc77s1OYoGgSdMH/iThr8LBQ0hf77tSusTDQXwS J15GU8Om0DAEGAN5RzJ6rYHi6nfj0DQBh0X6a5B9I8qcsVqCIDrtcA8MAfhuj+Yw Uis6hV+7GVQOuaCwAq5IjM+0XfjDXjXQjTbqPs+LDIBlodjLimP04oZ0uDyGExEo YosMof99IAE6+EjMdgsqeCzQEoZHmWiKPS8y8nTRU6UnF4ucY8E/vM6mAZuSh+VY DML6EC3YqDsWo+1aSxeVk122JBx+6zPEjPUTGvMQcJLbve5JmSLDLMvdmLZfsecy oUsRW0JuckxaH0gVMVGtCy8Wz4GjZf/2TuOLyGudj2JwHCDutAbXbXd12qgbiZo= =zo1J -----END PGP SIGNATURE----- From ixmatus@REDACTED Wed Jul 20 07:08:00 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Tue, 19 Jul 2011 22:08:00 -0700 Subject: [erlang-questions] Process pool map/3 implementation Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 For a work project I have a large list (thousands of items) to process and at first built a "pmap" implementation as per Joe's book until I found the plists module (which is awesome btw). There is one glaring issue with the list -> subdivide -> spawn x processes for n sublist items strategy; if an item in the sublist takes longer than all the other items it blocks the entire resource allotment until it is done. In most cases, the plists/pmap implementation works just fine because the items in the list probably don't take more than a few milliseconds to map the fun over. However, it does become an issue when that is not the case. So, I figured the next best strategy would be to implement a process pool since it would allow for slow running processes to continue their work while finished processes can die and new processes spawned into the pool ready for work - so none of the resources are sitting idle. Right now, my module isn't nearly as feature-complete as the plists module is - this is only a drop in replacement for map. Please submit your criticisms and comments to me at this address. You may find the code on BitBucket: https://bitbucket.org/ixmatus/ppool - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= =GP7K -----END PGP SIGNATURE----- From dmitrii@REDACTED Wed Jul 20 10:07:32 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 20 Jul 2011 11:07:32 +0300 Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: References: Message-ID: <47FA44AD-D56F-4A82-A9AF-70F3F02CFD70@dmitriid.com> There's also RabbitMQ's worker_pool, http://www.lshift.net/blog/2010/03/29/on-the-limits-of-concurrency-worker-pools-in-erlang, and my feeble attempt at somewhat extending it: https://github.com/dmitriid/worker_pool > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > For a work project I have a large list (thousands of items) to process > and at first built a "pmap" implementation as per Joe's book until I > found the plists module (which is awesome btw). > > There is one glaring issue with the list -> subdivide -> spawn x > processes for n sublist items strategy; if an item in the sublist takes > longer than all the other items it blocks the entire resource allotment > until it is done. > > In most cases, the plists/pmap implementation works just fine because > the items in the list probably don't take more than a few milliseconds > to map the fun over. However, it does become an issue when that is not > the case. > > So, I figured the next best strategy would be to implement a process > pool since it would allow for slow running processes to continue their > work while finished processes can die and new processes spawned into the > pool ready for work - so none of the resources are sitting idle. > > Right now, my module isn't nearly as feature-complete as the plists > module is - this is only a drop in replacement for map. Please submit > your criticisms and comments to me at this address. > > You may find the code on BitBucket: https://bitbucket.org/ixmatus/ppool > > - -- > Parnell "ixmatus" Springmeyer (http://ixmat.us) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > =GP7K > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Wed Jul 20 12:04:06 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 20 Jul 2011 12:04:06 +0200 Subject: [erlang-questions] Upgrade from R13B02 to R14B03 In-Reply-To: References: Message-ID: Hi Allen! I have to be honest and say I don't really know how this is supposed to work. I know that the appup files in kernel and stdlib are intentionally left empty, since it is never an option to hot-swap code from these applications - i.e. the emulator must always be restarted. However, it is of course not ok that systools:make_relup fails due to this! I will investigate further, but I can not promise a quick answer since most of my colleagues are on vacation and I definitely need to discuss this... It would be interesting though, if it is possible, to see your fixed .appup-files and the resulting relup that caused the second problem with code loading. Regards /siri@REDACTED 2011/7/19 Allen Kim > Did make all .appup files for all system applications. I don't see the > previous errors any more. > Then, installed erlang otp r14b03 to target machine > and when I run release_handler:install_release/1 I see the following error. > > =ERROR REPORT==== 19-Jul-2011::14:15:36 === > Loading of > /home/tracking/projects/tracking/erlang/lib/inets-5.6/ebin/httpd_manager.beam > failed: badfile > > =ERROR REPORT==== 19-Jul-2011::14:15:36 === > beam/beam_load.c(1300): Error loading module httpd_manager: > use of opcode 151; this emulator supports only up to 148 > > > =ERROR REPORT==== 19-Jul-2011::14:15:36 === > beam/beam_load.c(1300): Error loading module tftp_engine: > use of opcode 151; this emulator supports only up to 148 > > > =ERROR REPORT==== 19-Jul-2011::14:15:36 === > Loading of > /home/tracking/projects/tracking/erlang/lib/inets-5.6/ebin/tftp_engine.beam > failed: badfile > > anyone? > > From: Allen Kim > Date: Thu, 14 Jul 2011 15:31:22 -0500 > To: "erlang-questions@REDACTED" > Cc: Allen Kim > Subject: Upgrade from R13B02 to R14B03 > > Hi, > > I have installed elrang otp R14B03 on my machine. > Then, I wanted to upgrade erlang otp from R13B02 to R14B03. > > I have myapp-1.9.0.rel like this: > > {release,{"tracking","1.9.0"},*{erts,"5.7.3"},* > [ > {kernel,"2.13.3"}, {sasl,"2.1.7"}, {stdlib,"1.16.3"}, > {compiler, "4.6.3", load}, {crypto, "1.6.1", permanent}, > {edoc,"0.7.6.4", load}, {et,"1.3.2"}, > {syntax_tools,"1.6.3", load}, {gs,"1.5.10"}, {inets,"5.1.3"}, > {mnesia,"4.4.11", permanent}, {observer,"0.9.8.1"}, {os_mon,"2.2.3", > permanent}, > {otp_mibs,"1.0.5", load}, {snmp,"4.13.5", load}, > {public_key,"0.3", load}, {ssl,"3.10.4", load}, {tools,"2.6.4", > load}, {webtool,"0.8.4", load}, > {xmerl,"1.2.1", load}, {myapp,"1.0.0", load} > ]}. > > and, I have myapp-2.0.0.rel like this: > > {release,{"tracking","2.0.0"},*{erts,"5.8.4"},* > [ > {kernel,"2.14.4"}, {sasl,"2.1.9.4"}, {stdlib,"1.17.4"}, > {compiler, "4.7.4", load}, {crypto, "2.0.3", permanent}, > {edoc,"0.7.8", load}, {et,"1.4.3"}, > {syntax_tools,"1.6.7.1", load}, {gs,"1.5.13"}, {inets,"5.6"}, > {mnesia,"4.4.19", permanent}, {observer,"0.9.9"}, {os_mon,"2.2.6", > permanent}, > {otp_mibs,"1.0.6", load}, {snmp,"4.20", load}, {public_key,"0.12", > load}, {ssl,"4.1.5", load}, {tools,"2.6.6.4", load}, > {webtool,"0.8.8", load}, > {xmerl,"1.2.9", load}, {myapp,"1.0.0", load} > ]}. > > when I run make_relup command, I see the following error. > > error = systools:make_relup("myapp-2.0.0", ["myapp-1.9.0"], > ["myapp-1.9.0"], [restart_emulator, {path, > ["/home/allen/releases/1.9.0/lib/*/ebin", > "/home/allen/applications/*/ebin"]}]) > No release upgrade script entry for kernel-2.14.4 to kernel-2.13.3 in file > "/home/azoogle/projects/releases/1.9.0/lib/kernel-2.14.4/ebin/kernel.appup" > > that appup file looks like this. > $ cat > /home/azoogle/projects/releases/1.9.0/lib/kernel-2.14.4/ebin/kernel.appup > {"2.14.4",[],[]}. > > Does it mean do I have to write all .appup files for all application that I > want to upgrade with erlang version upgrade? > > Is there any one who upgraded erlang version with systools:make_relup? > > Thanks for your reply in advance > > Allen. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stamarit@REDACTED Wed Jul 20 17:07:36 2011 From: stamarit@REDACTED (Salvador Tamarit) Date: Wed, 20 Jul 2011 17:07:36 +0200 Subject: [erlang-questions] Statically determine values of variables Message-ID: Hi, Do you know any static analysis (a module, a tool or at least a paper) able to determine the (possible) values that a given variable can take during execution? Thanks. Salva From ixmatus@REDACTED Wed Jul 20 18:55:24 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Wed, 20 Jul 2011 09:55:24 -0700 Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: <47FA44AD-D56F-4A82-A9AF-70F3F02CFD70@dmitriid.com> (Dmitrii Dimandt's message of "Wed, 20 Jul 2011 11:07:32 +0300") References: <47FA44AD-D56F-4A82-A9AF-70F3F02CFD70@dmitriid.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for pointing that out - I tried looking for stuff that was already out there but couldn't find anything. Suppose I should have asked first!! Dmitrii Dimandt writes: > There's also RabbitMQ's worker_pool, http://www.lshift.net/blog/2010/03 > /29/on-the-limits-of-concurrency-worker-pools-in-erlang, and my feeble > attempt at somewhat extending it: https://github.com/dmitriid/ > worker_pool > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > For a work project I have a large list (thousands of items) to > process > and at first built a "pmap" implementation as per Joe's book until > I > found the plists module (which is awesome btw). > > There is one glaring issue with the list -> subdivide -> spawn x > processes for n sublist items strategy; if an item in the sublist > takes > longer than all the other items it blocks the entire resource > allotment > until it is done. > > In most cases, the plists/pmap implementation works just fine > because > the items in the list probably don't take more than a few > milliseconds > to map the fun over. However, it does become an issue when that is > not > the case. > > So, I figured the next best strategy would be to implement a > process > pool since it would allow for slow running processes to continue > their > work while finished processes can die and new processes spawned > into the > pool ready for work - so none of the resources are sitting idle. > > Right now, my module isn't nearly as feature-complete as the plists > module is - this is only a drop in replacement for map. Please > submit > your criticisms and comments to me at this address. > > You may find the code on BitBucket: https://bitbucket.org/ixmatus/ > ppool > > - -- > Parnell "ixmatus" Springmeyer (http://ixmat.us) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > =GP7K > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > =================================== > Dmitrii Dimandt > dmitrii@REDACTED > > ------------------------------------------------------------ > Erlang in Russian > http://erlanger.ru/ > > TurkeyTPS > http://turkeytps.com/ > ------------------------------------------------------------ > > LinkedIn: http://www.linkedin.com/in/dmitriid > GitHub: https://github.com/dmitriid > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJOJwh8AAoJEPvtlbpI1POLeEoIAJClEPTB6onqGk/1uPvZNW6M KQNYEjFTr3gm5ZP7oiQNcdlLbjSoOXtHMok6eZxOz2sKzNOiIcDa+7pbEbVvybPD 8WncYqKMS65CQQNMyG5AXyIOtRuCygLQgbSeSUEjuAprGUhTid9aklIeCIDMY1d6 tmdleHZZhvBKtONxnFtKJ7u+a2CzywNRBZ4BaKB+ThfvMnjNSlEM1IswujV8T6by vuld6eHtZu6410ksM05PK6FiDObdUa728/3E6BSz16ZSag0QYafC9BDHJ0S11eSS uazgs8Au6VK94tKLm/XE8f1JSNBMRKknWtbfuAHKZd46TLtI0KCntaAaT5VLAWk= =7RDO -----END PGP SIGNATURE----- From daniel.goertzen@REDACTED Wed Jul 20 20:36:20 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 20 Jul 2011 13:36:20 -0500 Subject: [erlang-questions] Statically determine values of variables In-Reply-To: References: Message-ID: The "typer" tool will show the inferred types of all the functions in a module. Try adding the variable in question to the return value of the function and then see what typer says. Cheers, Dan. On Wed, Jul 20, 2011 at 10:07 AM, Salvador Tamarit wrote: > Hi, > > Do you know any static analysis (a module, a tool or at least a paper) > able to determine the (possible) values that a given variable can take > during execution? > > Thanks. > Salva > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethstone@REDACTED Wed Jul 20 22:47:42 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Wed, 20 Jul 2011 15:47:42 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: I wonder if an ets solution wouldn't be as good, as these solutions are just going to find some way of holding the data in memory anyways. WIth ets, you can do things like hold compiled erlydtl/mustache templates inside of it... Kenny On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: > If you're already using Nginx and just want control over URLs, you > have access to the standard rewrite module: > > http://wiki.nginx.org/HttpRewriteModule > > This is not 30x redirection btw, unless of course you want that. > > On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter > wrote: > > Sam, Jesper both these sound great, thanks! I'm also going to look into > what > > Jack was saying about how Rails handles some stuff. > > But I'm leaning towards Jesper's idea with Varnish being the best... I'm > one > > of those people who scoff at most benchmarks so not sure I'll bother to > do > > one for this, but maybe, if someone here can A) suggest the best setup > for > > it B) if it makes sense at all or would just be another worthless > benchmark > > that really gives no information about reality. > > Thanks! > > Tristan > > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen > > wrote: > >> > >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter > >> wrote: > >> > >> > Can anyone think of a way I can keep the nice URLs and serve the > static > >> > html > >> > files through nginx or another webserver. > >> > >> Put a Varnish accelerator in front of your system > >> (https://www.varnish-cache.org/). That way, it doesn't matter if your > >> backend is slow at serving files as the accelerator will just cache > >> static stuff for you. In addition, you avoid the trouble of going > >> through another system as a proxy for static content. Also, the > >> solution is quite modular. On the development system, you don't need > >> more than a single system running Erlang. > >> > >> In my opinion, there is little reason not to plug into the whole > >> industry there is where the main point is to make serving HTTP go > >> faster. Trying to beat that with Erlang is probably possible, but I > >> don't think it is beneficial. Varnish is really really hard to beat. > >> It is built specifically for being insanely fast and it serves its > >> data from a shared mmap()'ing, scales to multiple CPUs and is a big > >> blob of nasty C code. I'd rather stand on the shoulders here than > >> trying to mess with it myself. > >> > >> > >> -- > >> J. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Wed Jul 20 23:19:17 2011 From: andrew@REDACTED (Andrew Thompson) Date: Wed, 20 Jul 2011 17:19:17 -0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP Message-ID: <20110720211917.GA16695@hijacked.us> As part of the work I've been doing at Basho on Riak, I was asked to write a new logging framework that would allow us to improve the logging experience for users. Lager is the result of that work and I'm glad to announce it is available as open-source under the Apache 2.0 licence for anyone to use. I've written up a blog post over at the Basho blog going into more details: http://blog.basho.com/2011/07/20/Introducing-Lager-A-New-Logging-Framework-for-ErlangOTP/ You can also take a look at the code and the README over at the github repo: https://github.com/basho/lager Please let me know what you think, I'm open to suggestions on improving and enhancing it. Pull requests would be great too. Andrew From essen@REDACTED Thu Jul 21 01:29:27 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 21 Jul 2011 01:29:27 +0200 Subject: [erlang-questions] Anyone using PropEr on their project yet? Message-ID: <4E2764D7.4070505@dev-extend.eu> Hello, I'm wondering if anyone has been using PropEr on their project yet and if so how you've set it up for optimal workflow. I'm guessing a ct suite with properties attached should work great (I think PropEr itself is doing this too) but someone might have gotchas or other interesting tips. If anyone has setup PropEr tests in their open source project I'd really like to take a look. -- Lo?c Hoguin Dev:Extend From tristan.sloughter@REDACTED Thu Jul 21 01:38:17 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 20 Jul 2011 18:38:17 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Kenny, yeah, thats what I was thinking of doing as a cache method if I couldn't use something easily in front of Webmachine like Varnish (didn't actually think of Varnish until Jesper brought it up. Maybe a bit of both... Since while I don't want to use any templating on the backend, I want to end up with a general web framework from all this. Tristan On Wed, Jul 20, 2011 at 3:47 PM, Kenny Stone wrote: > I wonder if an ets solution wouldn't be as good, as these solutions are > just going to find some way of holding the data in memory anyways. WIth > ets, you can do things like hold compiled erlydtl/mustache templates inside > of it... > > Kenny > > > On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: > >> If you're already using Nginx and just want control over URLs, you >> have access to the standard rewrite module: >> >> http://wiki.nginx.org/HttpRewriteModule >> >> This is not 30x redirection btw, unless of course you want that. >> >> On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter >> wrote: >> > Sam, Jesper both these sound great, thanks! I'm also going to look into >> what >> > Jack was saying about how Rails handles some stuff. >> > But I'm leaning towards Jesper's idea with Varnish being the best... I'm >> one >> > of those people who scoff at most benchmarks so not sure I'll bother to >> do >> > one for this, but maybe, if someone here can A) suggest the best setup >> for >> > it B) if it makes sense at all or would just be another worthless >> benchmark >> > that really gives no information about reality. >> > Thanks! >> > Tristan >> > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen >> > wrote: >> >> >> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >> >> wrote: >> >> >> >> > Can anyone think of a way I can keep the nice URLs and serve the >> static >> >> > html >> >> > files through nginx or another webserver. >> >> >> >> Put a Varnish accelerator in front of your system >> >> (https://www.varnish-cache.org/). That way, it doesn't matter if your >> >> backend is slow at serving files as the accelerator will just cache >> >> static stuff for you. In addition, you avoid the trouble of going >> >> through another system as a proxy for static content. Also, the >> >> solution is quite modular. On the development system, you don't need >> >> more than a single system running Erlang. >> >> >> >> In my opinion, there is little reason not to plug into the whole >> >> industry there is where the main point is to make serving HTTP go >> >> faster. Trying to beat that with Erlang is probably possible, but I >> >> don't think it is beneficial. Varnish is really really hard to beat. >> >> It is built specifically for being insanely fast and it serves its >> >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >> >> blob of nasty C code. I'd rather stand on the shoulders here than >> >> trying to mess with it myself. >> >> >> >> >> >> -- >> >> J. >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jul 21 01:48:35 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 20 Jul 2011 16:48:35 -0700 Subject: [erlang-questions] Anyone using PropEr on their project yet? In-Reply-To: <4E2764D7.4070505@dev-extend.eu> References: <4E2764D7.4070505@dev-extend.eu> Message-ID: I have some in kvc: https://github.com/etrepum/kvc On Wed, Jul 20, 2011 at 4:29 PM, Lo?c Hoguin wrote: > Hello, > > I'm wondering if anyone has been using PropEr on their project yet and > if so how you've set it up for optimal workflow. I'm guessing a ct suite > with properties attached should work great (I think PropEr itself is > doing this too) but someone might have gotchas or other interesting tips. > > If anyone has setup PropEr tests in their open source project I'd really > like to take a look. > > -- > Lo?c Hoguin > Dev:Extend > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From kennethstone@REDACTED Thu Jul 21 01:51:33 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Wed, 20 Jul 2011 18:51:33 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: One way to look at it is that Erlang has (more powerful) versions of redis/memcached built in with ETS and Mnesia. These are pretty nice features for web development, and I'm always a fan of cutting down the external dependencies for deployment and dev. Just rebar your repo and you get this powerful, self-contained web server executable (one of the selling points of couchdb, actually). Kenny On Wed, Jul 20, 2011 at 6:38 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Kenny, yeah, thats what I was thinking of doing as a cache method if I > couldn't use something easily in front of Webmachine like Varnish (didn't > actually think of Varnish until Jesper brought it up. Maybe a bit of both... > Since while I don't want to use any templating on the backend, I want to end > up with a general web framework from all this. > > Tristan > > > On Wed, Jul 20, 2011 at 3:47 PM, Kenny Stone wrote: > >> I wonder if an ets solution wouldn't be as good, as these solutions are >> just going to find some way of holding the data in memory anyways. WIth >> ets, you can do things like hold compiled erlydtl/mustache templates inside >> of it... >> >> Kenny >> >> >> On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: >> >>> If you're already using Nginx and just want control over URLs, you >>> have access to the standard rewrite module: >>> >>> http://wiki.nginx.org/HttpRewriteModule >>> >>> This is not 30x redirection btw, unless of course you want that. >>> >>> On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter >>> wrote: >>> > Sam, Jesper both these sound great, thanks! I'm also going to look into >>> what >>> > Jack was saying about how Rails handles some stuff. >>> > But I'm leaning towards Jesper's idea with Varnish being the best... >>> I'm one >>> > of those people who scoff at most benchmarks so not sure I'll bother to >>> do >>> > one for this, but maybe, if someone here can A) suggest the best setup >>> for >>> > it B) if it makes sense at all or would just be another worthless >>> benchmark >>> > that really gives no information about reality. >>> > Thanks! >>> > Tristan >>> > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen >>> > wrote: >>> >> >>> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >>> >> wrote: >>> >> >>> >> > Can anyone think of a way I can keep the nice URLs and serve the >>> static >>> >> > html >>> >> > files through nginx or another webserver. >>> >> >>> >> Put a Varnish accelerator in front of your system >>> >> (https://www.varnish-cache.org/). That way, it doesn't matter if your >>> >> backend is slow at serving files as the accelerator will just cache >>> >> static stuff for you. In addition, you avoid the trouble of going >>> >> through another system as a proxy for static content. Also, the >>> >> solution is quite modular. On the development system, you don't need >>> >> more than a single system running Erlang. >>> >> >>> >> In my opinion, there is little reason not to plug into the whole >>> >> industry there is where the main point is to make serving HTTP go >>> >> faster. Trying to beat that with Erlang is probably possible, but I >>> >> don't think it is beneficial. Varnish is really really hard to beat. >>> >> It is built specifically for being insanely fast and it serves its >>> >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >>> >> blob of nasty C code. I'd rather stand on the shoulders here than >>> >> trying to mess with it myself. >>> >> >>> >> >>> >> -- >>> >> J. >>> > >>> > >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Thu Jul 21 01:59:05 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Wed, 20 Jul 2011 18:59:05 -0500 Subject: [erlang-questions] Anyone using PropEr on their project yet? In-Reply-To: <4E2764D7.4070505@dev-extend.eu> References: <4E2764D7.4070505@dev-extend.eu> Message-ID: Erlware commons (https://github.com/ericbmerritt/erlware_commons/tree/next) is using it heavily for the new signature modules, and its slowly coming into use for sinan. On Wed, Jul 20, 2011 at 6:29 PM, Lo?c Hoguin wrote: > Hello, > > I'm wondering if anyone has been using PropEr on their project yet and > if so how you've set it up for optimal workflow. I'm guessing a ct suite > with properties attached should work great (I think PropEr itself is > doing this too) but someone might have gotchas or other interesting tips. > > If anyone has setup PropEr tests in their open source project I'd really > like to take a look. > > -- > Lo?c Hoguin > Dev:Extend > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu Jul 21 03:41:59 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 21 Jul 2011 01:41:59 +0000 (GMT) Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: Message-ID: <218172986.121321311212519505.JavaMail.root@zimbra> One quick question: what was wrong with the straightforward solution of just spawning one process for each element in the list? Did this break or do you actually need more control? Robert ----- "Parnell Springmeyer" wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > For a work project I have a large list (thousands of items) to > process > and at first built a "pmap" implementation as per Joe's book until I > found the plists module (which is awesome btw). > > There is one glaring issue with the list -> subdivide -> spawn x > processes for n sublist items strategy; if an item in the sublist > takes > longer than all the other items it blocks the entire resource > allotment > until it is done. > > In most cases, the plists/pmap implementation works just fine because > the items in the list probably don't take more than a few > milliseconds > to map the fun over. However, it does become an issue when that is > not > the case. > > So, I figured the next best strategy would be to implement a process > pool since it would allow for slow running processes to continue > their > work while finished processes can die and new processes spawned into > the > pool ready for work - so none of the resources are sitting idle. > > Right now, my module isn't nearly as feature-complete as the plists > module is - this is only a drop in replacement for map. Please submit > your criticisms and comments to me at this address. > > You may find the code on BitBucket: > https://bitbucket.org/ixmatus/ppool > > - -- > Parnell "ixmatus" Springmeyer (http://ixmat.us) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > =GP7K > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Thu Jul 21 07:50:56 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 21 Jul 2011 09:50:56 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: <20110720211917.GA16695@hijacked.us> References: <20110720211917.GA16695@hijacked.us> Message-ID: I will not get tired to tell, that current erlang implementations of logging are great OOM killers for cases, when you need to store 20+ megabytes of data in process. OTP guys have added good format_state method, but one bad place has left: it is dumping reason. And reason of crash may include whole process state. I have fixed gen_server for this: https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/gen_server_ems.erl#L747 and rewritten pretty_printer: https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/io_lib_pretty_limited.erl Without these changes small problem could bring down whole system, making insolvent all claims for bullet-proof software. The same situation is with message queue. If any streaming process in erlyvideo blocks just for 5-20 seconds, its message queue will grow to several megabytes of data. Dumping it via loggers will finish erlang VM. But good one-line loggers are ok, especially for startup. As here: https://github.com/erlyvideo/erlyvideo/blob/master/apps/log4erl/src/error_logger_log4erl_h.erl#L72 I had to rewrite application startup handling so that it should be more readable. So, thanks for your code and I'm running fast to read and test it. I'll be glad if some of my code would be usefull for you. From max.lapshin@REDACTED Thu Jul 21 07:53:59 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 21 Jul 2011 09:53:59 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: And, Andrew. Log rotation is the _most_ important feature in any logger. In fact, any logging system must be started from log rotator, because as a rule of thumb, it should be considered, that nobody ever setups logrotate. One of my famous tools is runit for its log rotation: it stores all debug output to logs and you should never bother about space on disks. From tony.arcieri@REDACTED Thu Jul 21 09:39:45 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 00:39:45 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang Message-ID: This is an idea that's been bouncing around in my head for awhile now and I thought I'd just bust it out: https://gist.github.com/1096711 Raw patch available here (I just grabbed the latest source code off Github to make this): https://raw.github.com/gist/1096711/ec763d3440fe1e1c07579e8ff78b3cdbc04e87e0/funargs.patch -- There's lots of cases where you just want to pass a single fun as an argument to a function. By convention this fun is typically passed as the first argument. However, the problem with Erlang's fun syntax is, well, it isn't fun. It's kind of noisy, and passing a fun as an argument makes it even noisier. Ruby has this really nifty feature called blocks which, to put not too fine a point on it, provides some awesome syntactic sugar for passing an anonymous function as an argument to another function. In Ruby, blocks look like this: ? ? [1,2,3,4,5].map do |n| ? ? ? ? n * 2 ? ? end (there are other forms, but this is the one I'm focusing on) Ask any Rubyist and they'll tell you blocks are the bee's knees. They are constantly trotted out as one of the most awesome and amazing features of Ruby. Many attempts have been made to add blocks to languages like Python (e.g. http://tav.espians.com/ruby-style-blocks-in-python.html) but it doesn't work in Python due to quirks in its indent-based grammar. However, I'd rather ask: can Erlang have something like Ruby like blocks? Yes, yes it can. -- The linked patch implements a feature I'm tentatively calling "funargs" (yes, I know, this name has prior usage in the funarg problem. If you don't like it suggest a better one!) The patch adds a new 'do' keyword and otherwise copies the Ruby syntax outright. Let's compare what Erlang lets you do now with what the syntax this patch provides allows. Before: mnesia:transaction(fun() -> ? ?mnesia:write(#thing{foo=1, bar=2, baz=3}) end). After: mnesia:transaction do ?? mnesia:write(#thing{foo=1, bar=2, baz=3}) end. That's marginally better. How about some more examples? Before: lists:map(fun(N) -> N * 2 end, [1,2,3,4,5]). After: lists:map([1,2,3,4,5]) do |N| N * 2 end. Again, it's marginally better. What if there are more arguments? Before: lists:foldl(fun(Octec, Acc) -> Acc bsl 8 bor Octet end, 0, [127,0,0,1]). After: lists:foldl(0, [127,0,0,1]) do |Octet, Acc| Acc bsl 8 bor Octet end. In this case I definitely prefer the latter form. Try to imagine functions with much more expressions in the fun body. So I'll admit so far none of these examples are *particularly* more compelling than their equivalent "fun" forms. Where blocks get really interesting is when you nest them, particularly when building what Rubyists refer to as DSLs. Below is a hypothetical example I think could actually be implemented using this syntax, perhaps with a custom error handler, which I admit would be a complete hack, but I think it'd be pretty interesting. I tried to translate an example builder template from Ruby available here: http://danengle.us/2009/05/generating-custom-xml-for-your-rails-app/ Let's imagine we have a custom error handler and a parameterized module "Xml" which is thunking to a process that's building an XML document for us on the fly. We could build XML in Erlang as follows: Xml:posts do lists:each(Posts) do |Post| Xml:post do Xml:title(Post#post.title), Xml:body(Post#post.body), Xml:published_at(Post#post.published_at), Xml:comments do lists:each(Post#post.comments) do |Comment| Xml.comment do Xml:body(Comment#comment.body) end end end end end end And voila, you have a fairly decent DSL for building XML from Erlang. Thoughts? -- Tony Arcieri From tony.arcieri@REDACTED Thu Jul 21 09:54:14 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 00:54:14 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: > And voila, you have a fairly decent DSL for building XML from Erlang. For comparison, here's the hypothetical XML "DSL" in regular Erlang syntax versus the block form: https://gist.github.com/1096754 -- Tony Arcieri From vladdu55@REDACTED Thu Jul 21 10:58:29 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 21 Jul 2011 10:58:29 +0200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: Hi! On Thu, Jul 21, 2011 at 09:39, Tony Arcieri wrote: > There's lots of cases where you just want to pass a single fun as an > argument to a function. By convention this fun is typically passed as > the first argument. However, the problem with Erlang's fun syntax is, > well, it isn't fun. It's kind of noisy, and passing a fun as an > argument makes it even noisier. > > Ruby has this really nifty feature called blocks which, to put not too > fine a point on it, provides some awesome syntactic sugar for passing > an anonymous function as an argument to another function. I think it is useful to investigate this area. The verbosity of function definitions is not pleasant, but I would argue that it is mostly necessary (give or take a few tokens). The reason for that is that we also have guards and multiple clauses. Your new syntax helps for the simple cases, but the general case won't be much less verbose than the current syntax. Or at least I can't see how it could be simplified, maybe you can. An argument can be made that the simple cases should be easier to type and read, but then there are two ways to express the same thing. Also, if a code modification leads to the function being no longer a 'simple case', then the whole expression needs to be rewritten to use the full syntax. How do you handle guards and multiple clauses in Reia? One could just as well instead of "do |X, Y|" write "fun (X, Y) ->" and keep the existing syntax for the fun. This will allow guards and multiple clauses. In that case, the proposed change will be that we could write Mod:Fun(Arg1, Arg2) fun(X) -> foo(X) end, instead of Mod:Fun(Arg1, Arg2, fun(X) -> foo(X) end), which is not really a big difference... For your XML builder example, the lack of parentheses is the most compelling advantage, in my opinion. Having "do |X, Y|" instead of "fun (X, Y) ->" doesn't feel like a big deal. I'm not sure the "less parentheses" argument is enough - there are other aspects with Erlang that make in-language DSLs a less than pretty sight. regards, Vlad From jesper.louis.andersen@REDACTED Thu Jul 21 14:34:37 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 21 Jul 2011 14:34:37 +0200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: On Thu, Jul 21, 2011 at 09:39, Tony Arcieri wrote: > Ruby has this really nifty feature called blocks which, to put not too > fine a point on it, provides some awesome syntactic sugar for passing > an anonymous function as an argument to another function. In Ruby, > blocks look like this: > > ? ? [1,2,3,4,5].map do |n| > ? ? ? ? n * 2 > ? ? end Most of the trouble stems from a) No currying in Erlang b) A long anonymous function notation c) No sectioning In Haskell, your example is map (*2) [1..5] and if written without sectioning: map (\x -> x * 2) [1..5] and in SML which doesn't have a list constructor convenience notation: List.map (fn x => x * 2) [1,2,3,4,5] . Ruby blocks is a weak implementation of anonymous functions in every way. The problems from using lots of anonymous functions in Erlang programs is mainly because the notational overhead of using them is too big. But then again, I don't find lists:map(fun(X) -> X * 2 end, [1,2,3,4,5]), much harder to read. Yes, there is more clutter in there due to the function object. But it has a distinct advantage over the block in ruby which is that it is not statement-syntax but an expression. I can write foo(F) -> lists:map(F, [1,2,3,4,5]). and get away with it. In Ruby you have to do something more here to turn the do-block, which is a statement, into some kind of value you can then pass around. When I looked at Ruby in 2002 they had some "proc" objects to do this with. I am sure it is better today. My main gripe with the kind of notation you propose is that it doesn't *compose* in the sense of a mathematical algebra. It is also why I like currying, and why I absolutely *love* Agda 2 for its infix operator definitions. In Agda, you can do: data List (A : Set) : Set where [] : List A _?_ : A -> List A -> List A which defines that we have a list and that the notation of the cons is: x ? l, where x has type A and l has type List A. You can even do more advanced stuff like if_then_else to build if-then-else statements yourself. These tools allow you to construct algebraic solutions to nearly any problem statement syntax will help with. TL;DR I don't think we need a new kind of syntax for certain kinds of lambda sections (see the 'cut' proposal a couple of days ago from SRFI-26 in Scheme-land). But I do agree the notation is heavier than it could be. A more convenient notation would encourage people to use anonymous functions more. -- J. From dhananjay.nene@REDACTED Thu Jul 21 16:29:44 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Thu, 21 Jul 2011 19:59:44 +0530 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: On Thu, Jul 21, 2011 at 1:24 PM, Tony Arcieri wrote: >> And voila, you have a fairly decent DSL for building XML from Erlang. > > For comparison, here's the hypothetical XML "DSL" in regular Erlang > syntax versus the block form: > > https://gist.github.com/1096754 I couldn't but help being reminded of http://redmine.ruby-lang.org/issues/5054 when reading the gist of either version :) :) Dhananjay -- ---------------------------------------------------------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene google plus: http://gplus.to/dhananjaynene From max.lapshin@REDACTED Thu Jul 21 18:15:22 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 21 Jul 2011 20:15:22 +0400 Subject: [erlang-questions] How big can be number of ets tables? Message-ID: Erlang VM has option to limit number of ets tables. What bad will happen, if I set it to 65536? From tristan.sloughter@REDACTED Thu Jul 21 18:21:15 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 21 Jul 2011 11:21:15 -0500 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: How about doing that 'do' addition to make Erlando's use of Monads (especially the error monad that I love) look nicer. Instead of having a list. error_m do ... end ? On Thu, Jul 21, 2011 at 9:29 AM, Dhananjay Nene wrote: > On Thu, Jul 21, 2011 at 1:24 PM, Tony Arcieri > wrote: > >> And voila, you have a fairly decent DSL for building XML from Erlang. > > > > For comparison, here's the hypothetical XML "DSL" in regular Erlang > > syntax versus the block form: > > > > https://gist.github.com/1096754 > > I couldn't but help being reminded of > http://redmine.ruby-lang.org/issues/5054 when reading the gist of > either version :) :) > > Dhananjay > -- > > ---------------------------------------------------------------------------------------------------------------------------------- > http://blog.dhananjaynene.com twitter: @dnene google plus: > http://gplus.to/dhananjaynene > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Jul 21 21:13:27 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 20:13:27 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: > SRFI-26 in Scheme-land). But I do agree the notation is heavier than > it could be. A more convenient notation would encourage people to use > anonymous functions more. Interestingly, Haskellers are encouraged not to over-do anonymous functions is possible. This is probably because using sections and point-free style is more concise and arguably easier to follow. Currying and decent support for function composition would come way higher on my list of "want to have" stuff. From dbarker@REDACTED Thu Jul 21 21:29:12 2011 From: dbarker@REDACTED (Deryk Barker) Date: Thu, 21 Jul 2011 12:29:12 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <4E287E08.70804@camosun.bc.ca> Before the Ruby-istas get too boastful, arent' Ruby blocks essentially the same as Smalltalk blocks? deryk From tristan.sloughter@REDACTED Thu Jul 21 22:01:07 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 21 Jul 2011 15:01:07 -0500 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: A reason I end up with far more anonymous functions in my Erlang than Haskell code (and end up with longer functions in Erlang) is the lack of 'let' and 'where'. In Erlang I'd do: -spec area_of_circles(record(circle)) -> [float()] area_of_circles(Circles) -> lists:map(fun(C) -> math:pi() * math:pow(C#circle.radius, 2) end, Circles). While in Haskell: area_of_circles :: [Shape] -> [Float] area_of_circles circles = L.map (area_of_circle) circles where area_of_circle (Circle radius) = pi * radius^2 Not a great example, but I don't have time to come up with a better one... But maybe portrays what I mean about Erlang making me less likely to name anonymous functions and less likely to break things into smaller functions. Tristan On Thu, Jul 21, 2011 at 7:34 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Thu, Jul 21, 2011 at 09:39, Tony Arcieri > wrote: > > > Ruby has this really nifty feature called blocks which, to put not too > > fine a point on it, provides some awesome syntactic sugar for passing > > an anonymous function as an argument to another function. In Ruby, > > blocks look like this: > > > > [1,2,3,4,5].map do |n| > > n * 2 > > end > > Most of the trouble stems from > > a) No currying in Erlang > b) A long anonymous function notation > c) No sectioning > > In Haskell, your example is > > map (*2) [1..5] > > and if written without sectioning: > > map (\x -> x * 2) [1..5] > > and in SML which doesn't have a list constructor convenience notation: > > List.map (fn x => x * 2) [1,2,3,4,5] > > . > > Ruby blocks is a weak implementation of anonymous functions in every > way. The problems from using lots of anonymous functions in Erlang > programs is mainly because the notational overhead of using them is > too big. But then again, I don't find > > lists:map(fun(X) -> X * 2 end, [1,2,3,4,5]), > > much harder to read. Yes, there is more clutter in there due to the > function object. But it has a distinct advantage over the block in > ruby which is that it is not statement-syntax but an expression. I can > write > > foo(F) -> > lists:map(F, [1,2,3,4,5]). > > and get away with it. In Ruby you have to do something more here to > turn the do-block, which is a statement, into some kind of value you > can then pass around. When I looked at Ruby in 2002 they had some > "proc" objects to do this with. I am sure it is better today. My main > gripe with the kind of notation you propose is that it doesn't > *compose* in the sense of a mathematical algebra. It is also why I > like currying, and why I absolutely *love* Agda 2 for its infix > operator definitions. In Agda, you can do: > > data List (A : Set) : Set where > [] : List A > _?_ : A -> List A -> List A > > which defines that we have a list and that the notation of the cons > is: x ? l, where x has type A and l has type List A. You can even do > more advanced stuff like if_then_else to build if-then-else statements > yourself. These tools allow you to construct algebraic solutions to > nearly any problem statement syntax will help with. > > TL;DR I don't think we need a new kind of syntax for certain kinds of > lambda sections (see the 'cut' proposal a couple of days ago from > SRFI-26 in Scheme-land). But I do agree the notation is heavier than > it could be. A more convenient notation would encourage people to use > anonymous functions more. > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Jul 21 22:01:36 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 21:01:36 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: On 21 July 2011 06:53, Max Lapshin wrote: > And, Andrew. > Log rotation is the _most_ important feature in any logger. In fact, > any logging system must be started from log rotator, because as a rule > of thumb, it should be considered, that nobody ever setups logrotate. Yes, that's why https://github.com/hyperthunk/fastlog uses error_logger by default, which can benefit from sasl rolling log configuration and riak_err to minimise memory usage. Andrew - I find it interesting that you did the opposite, and optionally redirected error_logger to Lager! :) I like the way you check the levels though - https://github.com/basho/lager/blob/master/src/lager_console_backend.erl#L42 - this is cleaner than the boilerplate I ended up with in fastlog, which checks the atoms for all the possible levels. I might steal that. :) Do you support (or plan to support) named loggers? This is a useful feature, as well as the ability to set different levels for different (named) logger processes, which I use quite often. I notice in your parse_transform that you log a bunch of useful stuff. I did something similar (see https://github.com/hyperthunk/fastlog/wiki/Parse-Transform-Support for details) although I allow patterns to be defined (per logger) to specify which bits actually appear in the logs - again something you might like to think about. Also in your parse_transform, you seem to log time information by default - this can be quite expensive and it might be better to make this optional!? One final note - lager_mochiglobal appears to be doing some scary things. Is this really the easiest way to check whether or not a log level is set properly, and why do you need to check the level at the call site if the logging process (i.e., the gen_event handler) is also doing this? Maybe I've flicked thorugh the code too quickly and misunderstood. Anyway - hope these comments are helpful, and thanks for the contribution to the community. We have at least 4 logging frameworks to choose from now! :) From watson.timothy@REDACTED Thu Jul 21 22:07:27 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 21:07:27 +0100 Subject: [erlang-questions] Anyone using PropEr on their project yet? In-Reply-To: References: <4E2764D7.4070505@dev-extend.eu> Message-ID: > On Wed, Jul 20, 2011 at 6:29 PM, Lo?c Hoguin wrote: >> Hello, >> >> I'm wondering if anyone has been using PropEr on their project yet and >> if so how you've set it up for optimal workflow. I'm guessing a ct suite >> with properties attached should work great (I think PropEr itself is >> doing this too) but someone might have gotchas or other interesting tips. >> >> If anyone has setup PropEr tests in their open source project I'd really >> like to take a look. >> https://github.com/hyperthunk/hamcrest-erlang From max.lapshin@REDACTED Thu Jul 21 22:18:54 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Jul 2011 00:18:54 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: > > I like the way you check the levels though - > https://github.com/basho/lager/blob/master/src/lager_console_backend.erl#L42 > - this is cleaner than the boilerplate I ended up with in fastlog, > which checks the atoms for all the possible levels. I might steal > that. :) > I suppose it is a bad idea to filter logs in the single process. It seems a very good idea like in log4erl to compile module when reading configuration and filter messages _before_ sending them to error_logger. It is better to filter with 24 cores, than with one =) From tony.arcieri@REDACTED Thu Jul 21 22:43:44 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 13:43:44 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <4E287E08.70804@camosun.bc.ca> References: <4E287E08.70804@camosun.bc.ca> Message-ID: On Thu, Jul 21, 2011 at 12:29 PM, Deryk Barker wrote: > Before the Ruby-istas get too boastful, arent' Ruby blocks essentially the > same as Smalltalk blocks? There are definitely similarities between Ruby and Smalltalk blocks and the former was inspired by the latter, however the syntax is different. I implemented the Ruby syntax here, hence the comparison to Ruby. Obviously what I've implemented here doesn't have the same semantics as either Ruby or Smalltalk blocks. It's just syntactic sugar (or syntactic vomit, depending on your opinion) -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Jul 21 22:44:33 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 21:44:33 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: On 21 July 2011 21:18, Max Lapshin wrote: >> >> I like the way you check the levels though - >> https://github.com/basho/lager/blob/master/src/lager_console_backend.erl#L42 >> - this is cleaner than the boilerplate I ended up with in fastlog, >> which checks the atoms for all the possible levels. I might steal >> that. :) >> > > I suppose it is a bad idea to filter logs in the single process. It > seems a very good idea like in log4erl to > compile module when reading configuration and filter messages _before_ > sending them to error_logger. > > It is better to filter with 24 cores, than with one =) > I don't agree that this is an either or thing. It is true that turning on and off the levels at compilation time is clearly superior in terms of performance, but it isn't much use when you're deployed in production and wish to turn up (or down) the logging levels for a temporary period. Filtering out certainly logging levels at compile time is quite easy - tuning to get good performance at runtime without sacrificing flexibility is harder. Besides, log4erl also supports changing log levels on the appenders at runtime, so it is also doing some work at runtime. Probably the most efficient way would be to bucket the loggers according to all the levels they are enabled for, which would carry a little time overhead in finding a logger (for a given level) and a little space overhead in storing the buckets. I do agree that doing as much work in the client as possible is good. I would move the formatting work (which in fastlog is usually undertaken by error_logger, although that can be overriden - to the client process, but I have state associated with the individual loggers. I've been thinking about having a single proc to hold all the various states and having the parse_transform generate code that gets the state back and doing the level checking and actually logging calls in the client. Currently fastlog attempts to do things in as light a fashion as possible, as you can read about on the wiki: https://github.com/hyperthunk/fastlog/wiki. I'm going to benchmark before making any significant changes - perhaps the various authors of logging frameworks could agree on a sensible set of benchmarks that we could all aspire to? From max.lapshin@REDACTED Thu Jul 21 22:54:34 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Jul 2011 00:54:34 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: On Fri, Jul 22, 2011 at 12:44 AM, Tim Watson wrote: > temporary period. Filtering out certainly logging levels at compile > time is quite easy - tuning to get good performance at runtime without > sacrificing flexibility is harder. Besides, log4erl also supports > changing log levels on the appenders at runtime, so it is also doing > some work at runtime. log4erl compiles filtering module in runtime. It really works. From andrew@REDACTED Thu Jul 21 23:21:42 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 21 Jul 2011 17:21:42 -0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: <20110721212142.GA9340@hijacked.us> Trying to reply to several emails here, bear with me. Lager actually filters in 2 places, at the logging callsite (via a lookup in mochiglobal, which is *very* fast) and then each backend also filters, because you may have different backends consuming different levels. Lager calculates the minimum level in use and stores that in the mochiglobal value. The use of mochiglobal was something I kind of agonized over, but its solid code (riak uses it heavily) and its the fastest option for this kind of 'read often, write seldom' value. For log rotation, I find the builtin rotation to be inadequate. Lager will shortly have its own size and date based rotation (size is done, but in a branch and I'm working on date right now) that should be more consistant with what normal rotation tools do. With regard to more configurable formatting, that's something I'd like to work on, but I don't want it to be too expensive. Maybe I can do some compile-time voodoo with like -DFORMAT_STRING="..." and use that to optimize how values are captured and formatted at compile time. Thanks for all the suggestions guys, I've certainly gotten some new ideas from this thread. Andrew From watson.timothy@REDACTED Thu Jul 21 23:50:01 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 22:50:01 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: <20110721212142.GA9340@hijacked.us> References: <20110720211917.GA16695@hijacked.us> <20110721212142.GA9340@hijacked.us> Message-ID: > With regard to more configurable formatting, that's something I'd like > to work on, but I don't want it to be too expensive. Maybe I can do some > compile-time voodoo with like -DFORMAT_STRING="..." and use that to > optimize how values are captured and formatted at compile time. So fastlog_util compiles its logging patterns at runtime into a fun, which does little more than extract certain record fields from the logging record (which the parse_transform produces) - and replace their values in the format string. That seemed fairly efficient to me. I'll have a look at mochiglobal - sounds interesting. From watson.timothy@REDACTED Thu Jul 21 23:52:11 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 21 Jul 2011 22:52:11 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: On 21 July 2011 21:54, Max Lapshin wrote: > On Fri, Jul 22, 2011 at 12:44 AM, Tim Watson wrote: >> temporary period. Filtering out certainly logging levels at compile >> time is quite easy - tuning to get good performance at runtime without >> sacrificing flexibility is harder. Besides, log4erl also supports >> changing log levels on the appenders at runtime, so it is also doing >> some work at runtime. > > log4erl compiles filtering module in runtime. It really works. > I'll take a closer look - sounds interesting. What put me off originally is that there is a heck of a lot of code there, which tends to mean there's a lot going on whereas I wanted minimal impact and control over blocking the client (or better, making the work in the client) or not. Fastlog is only 665 lines of code, as it does very little work to get from the call site to the underlying logging module (error_logger by default). From ok@REDACTED Fri Jul 22 00:48:58 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 10:48:58 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: Ruby blocks are a crippled perversion of Smalltalk blocks. And Smalltalk blocks are practically the same as the way functions were written in Wirth's language Euler. > [1,2,3,4,5].map do |n| > n * 2 > end in Smalltalk this is just (1 to: 5) collect: [:n | n * 2] The Smalltalk collection classes provide *oodles* of opportunities to use them, and no few of the methods require *two* blocks. For example, aCollection do: [:each | Transcript print: each] separatedBy: [Transcript space] which doesn't really fit well into Ruby syntax. Another example would have been try-catch, which _could_ have been a function: try(Set_Up, % () -> Info Tear_Down, % (Info) -> () Body, % (Info) -> Answer Handler_List) % [(Info, Exception) -> Answer] > However, I'd rather ask: can Erlang have something like Ruby like > blocks? Yes, yes it can. Well yes, it does. They are called funs. > > The linked patch implements a feature I'm tentatively calling > "funargs" (yes, I know, this name has prior usage in the funarg > problem. If you don't like it suggest a better one!) Almost ANYTHING, up to and including "tinklebats", would be better than twisting existing words into incompatible meanings. Since the Latin word "fungor" means "to occupy oneself with anything", "perform", "execute" (Cassell's Latin Dictionary), why not call them "fungors"? > The patch adds a new 'do' keyword and otherwise copies the Ruby syntax > outright. > Let's compare what Erlang lets you do now with what the > syntax this patch provides allows. > > Before: > > mnesia:transaction(fun() -> > mnesia:write(#thing{foo=1, bar=2, baz=3}) > end). > > After: > > mnesia:transaction do > mnesia:write(#thing{foo=1, bar=2, baz=3}) > end. > > That's marginally better. It's a very small margin, and possibly a negative one. Should it not be ([]) do [ ->] end => (fun () -> end[, ) or (fun -> end[, ) ? So should it not be mnesia:transaction() do mnesia:write(#thing{foo=1, bar=2, baz=3}) end for consistency? > How about some more examples? > > Before: > > lists:map(fun(N) -> > N * 2 > end, [1,2,3,4,5]). > > After: > > lists:map([1,2,3,4,5]) do |N| > N * 2 > end. > > Again, it's marginally better. This is where the margin turns negative. What has mapping to do with "DO"? And YEEK! The argument list here looks like nothing else in Erlang whatever! Surely surely surely Erlang should look like Erlang! lists:map([1,2,3,4,5]) do (N) -> N*2 end (As a Smalltalk programmer, I really wish that Ruby hadn't taken Smalltalk's syntax for *local variables* and twisted it to mean block parameters.) The vertical bar notation here is NOT a good fit for a language where function arguments involve pattern matching, and patterns can involve vertical bars. > What if there are more arguments? > > Before: > > lists:foldl(fun(Octec, Acc) -> > Acc bsl 8 bor Octet > end, 0, [127,0,0,1]). > > After: > > lists:foldl(0, [127,0,0,1]) do |Octet, Acc| > Acc bsl 8 bor Octet > end. > > In this case I definitely prefer the latter form. Try to imagine > functions with much more expressions in the fun body. Try to imagine function calls with many more funs in their arguments! By the way, it's now a couple of years since I proposed syntax like (Acc where Acc = 0 then Acl bsl 8 bor Octet for Octet <- [127,0,0,1]) the three patterns being '(' opt-generators-and-guards ')' '[' opt-generators-and-guards ']' '{' where opt-generators-and-guards '}' opt-where = 'where' {',' }... | = '=' ['then' ] opt-generators-and-guards = {'for' | '||'} {generator|guard} {',' {generator|guard}}... | This has roots in ISWIM, Scheme, Clean, and Erlang. > So I'll admit so far none of these examples are *particularly* more > compelling than their equivalent "fun" forms. Where blocks get really > interesting is when you nest them, particularly when building what > Rubyists refer to as DSLs. They did not invent the idea. I think Erick Sandewall may have been the first to write about embedded DSLs (in Lisp) back in the 70s. There have been two USENIX conferences about DSLs. > Below is a hypothetical example I think > could actually be implemented using this syntax, perhaps with a custom > error handler, which I admit would be a complete hack, but I think > it'd be pretty interesting. I tried to translate an example builder > template from Ruby available here: > > http://danengle.us/2009/05/generating-custom-xml-for-your-rails-app/ > > Let's imagine we have a custom error handler and a parameterized > module "Xml" which is thunking to a process that's building an XML > document for us on the fly. We could build XML in Erlang as follows: > > Xml:posts do > lists:each(Posts) do |Post| > Xml:post do > Xml:title(Post#post.title), > Xml:body(Post#post.body), > Xml:published_at(Post#post.published_at), > Xml:comments do > lists:each(Post#post.comments) do |Comment| > Xml.comment do > Xml:body(Comment#comment.body) > end > end > end > end > end > end > > And voila, you have a fairly decent DSL for building XML from Erlang. I have serious trouble reading this, and part of the reason is that I see do and cannot help but read that as "do for every element of ". This is reinforced by the fact that in Erlang *everything* that is a function call has a pair of parentheses; "Xml:posts" looks far more like a variable than like a function call. The thing is, we already _have_ a way of writing XML in Erlang: {posts, [ {post, [ {title, [Post#post.title]}, {body, [Post#post.body]}, {published_at, [Post#post.published_at]}, {comments, [ {comment, [Comment#comment.body]} || Comment <- Post#post.comments]}} || Post <- Posts]} which is 9 lines to your 16 with no language extensions whatever. Give me a data structure, bursting with the seeds of its own transformation, rather than a tangle of commands. Ruby is an imperative language working by side effects. What works well for Ruby doesn't work so well for Erlang. From ok@REDACTED Fri Jul 22 01:11:45 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 11:11:45 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <787580D9-2722-4471-840F-64D18CE3855F@cs.otago.ac.nz> On 22/07/2011, at 12:34 AM, Jesper Louis Andersen wrote: > TL;DR I don't think we need a new kind of syntax for certain kinds of > lambda sections (see the 'cut' proposal a couple of days ago from > SRFI-26 in Scheme-land). But I do agree the notation is heavier than > it could be. A more convenient notation would encourage people to use > anonymous functions more. The whole of Erlang syntax is "heavier" than it might be. I once devised a revised syntax which I called "hErlang". For example, this is a hErlang module: -module predict -export (predict/2) -import math:(pi/0 sqrt/1) -author "Richard A. O'Keefe" -date {2002,08,14} -sccs "08/16/02" predict X0 Ns = let {Xs,Ys} = odds_and_evens Ns, Np = length Xs, XBar = if Np == 0 => 0.0 else sum Xs / Np fi, YBar = if Np == 0 => 0.0 else sum Ys / Np fi, Beta1 = sum [(X-XBar)*(Y-YBar) for X,Y in Xs,Ys] / sum [(X-XBar)*(X-XBar) for X in Xs], Beta0 = YBar - Beta1*XBar, S = sqrt((sum [(Y-Beta1*X-Beta0)^2 for X,Y in Xs,Ys])/(Np-2)), R = sqrt((Np+1)/Np + (X-XBar)^2/sum [(X-XBar)^2 for X in Xs]), T70 = S * R * tfun 0.85 (Np-2), T90 = S * R * tfun 0.95 (Np-2), Y0 = Beta1*X0+Beta0 in {Y-T90,Y-T70,Y,Y+T70,Y+T90} odds_and_evens [X,Y|XYs] = let {Xs,Ys} = odds_and_evens XYs in {[X|Xs],[Y|Ys]} odds_and_evens _ = {[],[]} tfun Goal N = let G = (Goal-0.5) * (gam N / gam (N+1)) * sqrt (pi() * N), F = \X. (1+X*X/N)^(-0.5*(N+1)) in bsearch F G 1.0 8.0 gam 1 = sqrt (pi()) gam 2 = 1.0 gam (K+2) = (K/2.0) * gam K bsearch F G Lo Hi | Hi-Lo <= 0.5e-6 = (Lo+Hi)*0.5 = let M = (Lo + Hi) * 0.5, S = simpson F 0 M 1.0e-5 10 0.0 in if S < G => bsearch F G M Hi or S >= G => bsearch F G Lo M fi simpson F X0 X1 Eps N R0 = % 2 'where' lines shorter. let W = (X1 - X0) / (2 * N), R = ( 4.0 * sum [F(X0 + W * (2*I-1)) for I in [1..N]] + 2.0 * sum [F(X0 + W * (2*I) ) for I in [1..N-1]] + F X0 + F X1 ) * (W/3.0) in if abs (R-R0) <= Eps => R else simpson F X0 X1 Eps (N*2) R fi It looks a lot like Haskell, but it's Erlang semantics all the way. Not unlike LFE, in fact. I never bothered finishing this, so don't ask for the code. I just took it far enough to determine that the thing *could* be done and *did* produce a useful payoff (my test files were about 1/3rd shorter than normal Erlang). But incorporating records and binary patterns and all the other things seemed too much like work. So a syntactically leaner Erlang *is* possible. The one thing I found in that experiment that's really important here is that you CAN'T get there from here one step at a time. To make a really lean Erlang, you *have* to take a great big jump to a coherently designed whole. From watson.timothy@REDACTED Fri Jul 22 01:19:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 00:19:07 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <787580D9-2722-4471-840F-64D18CE3855F@cs.otago.ac.nz> References: <787580D9-2722-4471-840F-64D18CE3855F@cs.otago.ac.nz> Message-ID: On 22 July 2011 00:11, Richard O'Keefe wrote: > > On 22/07/2011, at 12:34 AM, Jesper Louis Andersen wrote: >> TL;DR I don't think we need a new kind of syntax for certain kinds of >> lambda sections (see the 'cut' proposal a couple of days ago from >> SRFI-26 in Scheme-land). But I do agree the notation is heavier than >> it could be. A more convenient notation would encourage people to use >> anonymous functions more. > > The whole of Erlang syntax is "heavier" than it might be. > I once devised a revised syntax which I called "hErlang". > For example, this is a hErlang module: > > -module predict > -export (predict/2) > -import math:(pi/0 sqrt/1) > -author "Richard A. O'Keefe" > -date ?{2002,08,14} > -sccs ?"08/16/02" > > predict X0 Ns = > ? ?let {Xs,Ys} = odds_and_evens Ns, > ? ? ? ?Np ? ? ?= length Xs, > ? ? ? ?XBar ? ?= if Np == 0 => 0.0 else sum Xs / Np fi, > ? ? ? ?YBar ? ?= if Np == 0 => 0.0 else sum Ys / Np fi, > ? ? ? ?Beta1 ? = sum [(X-XBar)*(Y-YBar) for X,Y in Xs,Ys] / > ? ? ? ? ? ? ? ? ?sum [(X-XBar)*(X-XBar) for X ? in Xs], > ? ? ? ?Beta0 ? = YBar - Beta1*XBar, > ? ? ? ?S ? ? ? = sqrt((sum [(Y-Beta1*X-Beta0)^2 for X,Y in Xs,Ys])/(Np-2)), > ? ? ? ?R ? ? ? = sqrt((Np+1)/Np + (X-XBar)^2/sum [(X-XBar)^2 for X in Xs]), > ? ? ? ?T70 ? ? = S * R * tfun 0.85 (Np-2), > ? ? ? ?T90 ? ? = S * R * tfun 0.95 (Np-2), > ? ? ? ?Y0 ? ? ?= Beta1*X0+Beta0 > ? ?in {Y-T90,Y-T70,Y,Y+T70,Y+T90} > > odds_and_evens [X,Y|XYs] = let {Xs,Ys} = odds_and_evens XYs in {[X|Xs],[Y|Ys]} > odds_and_evens _ ? ? ? ? = {[],[]} > > tfun Goal N = > ? ?let G = (Goal-0.5) * (gam N / gam (N+1)) * sqrt (pi() * N), > ? ? ? ?F = \X. (1+X*X/N)^(-0.5*(N+1)) > ? ?in bsearch F G 1.0 8.0 > > gam 1 = sqrt (pi()) > gam 2 = 1.0 > gam (K+2) = (K/2.0) * gam K > > bsearch F G Lo Hi | Hi-Lo <= 0.5e-6 = (Lo+Hi)*0.5 > ?= let M = (Lo + Hi) * 0.5, > ? ? ? ?S = simpson F 0 M 1.0e-5 10 0.0 > ? ?in ?if S < G ?=> bsearch F G M Hi > ? ? ? ?or S >= G => bsearch F G Lo M fi > > simpson F X0 X1 Eps N R0 = % 2 'where' lines shorter. > ? ?let W = (X1 - X0) / (2 * N), > ? ? ? ?R = ( 4.0 * sum [F(X0 + W * (2*I-1)) for I in [1..N]] > ? ? ? ? ? ?+ 2.0 * sum [F(X0 + W * (2*I) ?) for I in [1..N-1]] > ? ? ? ? ? ?+ F X0 + F X1 ) * (W/3.0) > ? ?in if abs (R-R0) <= Eps => R else simpson F X0 X1 Eps (N*2) R fi > > It looks a lot like Haskell, but it's Erlang semantics all the way. > Not unlike LFE, in fact. > > I never bothered finishing this, so don't ask for the code. > I just took it far enough to determine that the thing *could* be > done and *did* produce a useful payoff (my test files were about > 1/3rd shorter than normal Erlang). ?But incorporating records and > binary patterns and all the other things seemed too much like work. > > So a syntactically leaner Erlang *is* possible. ?The one thing I > found in that experiment that's really important here is that you > CAN'T get there from here one step at a time. ?To make a really > lean Erlang, you *have* to take a great big jump to a coherently > designed whole. > That actually looks really nice. From ok@REDACTED Fri Jul 22 01:19:15 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 11:19:15 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <6AA087DE-0F5B-48D5-9EE3-99A4B8A21C88@cs.otago.ac.nz> On 22/07/2011, at 2:29 AM, Dhananjay Nene wrote: > On Thu, Jul 21, 2011 at 1:24 PM, Tony Arcieri wrote: >>> And voila, you have a fairly decent DSL for building XML from Erlang. >> >> For comparison, here's the hypothetical XML "DSL" in regular Erlang >> syntax versus the block form: >> >> https://gist.github.com/1096754 > > I couldn't but help being reminded of > http://redmine.ruby-lang.org/issues/5054 when reading the gist of > either version :) :) BCPL: $(tag $( $( $( $( $)tag PL/I: L: do; do; do; do; end L; Interlisp: [foo (bar (ugh (zap] and that's just three languages I've used that had some way of abbreviating a lot of "end"s. Occam, Python, Haskell, and Clean are four languages that eliminate the ends in the first place. This is another example where you cannot patch your way to improvement, a whole new front end being needed. (The hErlang example in my previous message looks as though it relies on indentation, but it doesn't.) From watson.timothy@REDACTED Fri Jul 22 01:20:00 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 00:20:00 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: > Almost ANYTHING, up to and including "tinklebats", would be > better than twisting existing words into incompatible meanings. > Since the Latin word "fungor" means "to occupy oneself with > anything", "perform", "execute" (Cassell's Latin Dictionary), > why not call them "fungors"? No no, I think "tinklebats" is a rocking name! :) From watson.timothy@REDACTED Fri Jul 22 01:21:27 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 00:21:27 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <6AA087DE-0F5B-48D5-9EE3-99A4B8A21C88@cs.otago.ac.nz> References: <6AA087DE-0F5B-48D5-9EE3-99A4B8A21C88@cs.otago.ac.nz> Message-ID: > and that's just three languages I've used that had some way of > abbreviating a lot of "end"s. ?Occam, Python, Haskell, and > Clean are four languages that eliminate the ends in the first > place. > > This is another example where you cannot patch your way to > improvement, a whole new front end being needed. > > (The hErlang example in my previous message looks as though > it relies on indentation, but it doesn't.) > I actually rather like Python and Haskell's use of indentation - god forbid I don't want to start a debate about it though. From ok@REDACTED Fri Jul 22 01:28:51 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 11:28:51 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <058F40E8-E91F-4CD1-8B72-49D28C68D5BE@cs.otago.ac.nz> On 22/07/2011, at 8:01 AM, Tristan Sloughter wrote: > A reason I end up with far more anonymous functions in my Erlang than Haskell code (and end up with longer functions in Erlang) is the lack of 'let' and 'where'. > > In Erlang I'd do: > > -spec area_of_circles(record(circle)) -> [float()] > area_of_circles(Circles) -> > lists:map(fun(C) -> > math:pi() * math:pow(C#circle.radius, 2) > end, Circles). > Wouldn't you write area_of_circles(Circles) -> [math:pi()*math:pow(C#circle.radius, 2) || C <- Circles]. > While in Haskell: > > area_of_circles :: [Shape] -> [Float] > area_of_circles circles = > L.map (area_of_circle) circles > where > area_of_circle (Circle radius) = pi * radius^2 Wouldn't you write area_of_circles circles = [pi*r^2 | Circle r <- circles] The problem isn't a lack of 'let' and 'where'. That's solved just by making names distinct. The problem is that normal function definitions don't nest; while you can do area_of_circles(Cs) -> Area = fun (C) -> math:pi()*math:pow(C#Circle.radius, 2) end, lists:map(Area, Cs). you still have to use a 'fun' to do it. From tony.arcieri@REDACTED Fri Jul 22 01:34:35 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 16:34:35 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > > However, I'd rather ask: can Erlang have something like Ruby like > > blocks? Yes, yes it can. > > Well yes, it does. They are called funs. This sort of sentiment lacks any sense of aesthetics. It is akin to saying that a potato sack is the same thing as a suit coat because both provide the same basic function of covering your torso. Let me call out explicitly what is ugly about Erlang fun syntax: it combines a symbolic token "->" with a keyword token "end" instead of a matching pair of symbolic tokens (e.g. "{" ... "}" or even "->" ... ".") or a matching pair of keywords (e.g. "do" ... "end"). I think there are a lot of people in the Erlang community who are either completely oblivious to how that sort of thing harms the aesthetics of the language or willfully choose to ignore it. This makes the Erlang "fun" syntax awkward and clumsy and not particularly "fun", when really anonymous functions are a powerful concept and should be a pleasure to use. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Fri Jul 22 01:47:27 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 21 Jul 2011 18:47:27 -0500 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <058F40E8-E91F-4CD1-8B72-49D28C68D5BE@cs.otago.ac.nz> References: <058F40E8-E91F-4CD1-8B72-49D28C68D5BE@cs.otago.ac.nz> Message-ID: Yes, I said it was a poor example :) On Thu, Jul 21, 2011 at 6:28 PM, Richard O'Keefe wrote: > > On 22/07/2011, at 8:01 AM, Tristan Sloughter wrote: > > > A reason I end up with far more anonymous functions in my Erlang than > Haskell code (and end up with longer functions in Erlang) is the lack of > 'let' and 'where'. > > > > In Erlang I'd do: > > > > -spec area_of_circles(record(circle)) -> [float()] > > area_of_circles(Circles) -> > > lists:map(fun(C) -> > > math:pi() * math:pow(C#circle.radius, 2) > > end, Circles). > > > Wouldn't you write > > area_of_circles(Circles) -> > [math:pi()*math:pow(C#circle.radius, 2) || C <- Circles]. > > > While in Haskell: > > > > area_of_circles :: [Shape] -> [Float] > > area_of_circles circles = > > L.map (area_of_circle) circles > > where > > area_of_circle (Circle radius) = pi * radius^2 > > Wouldn't you write > > area_of_circles circles = [pi*r^2 | Circle r <- circles] > > The problem isn't a lack of 'let' and 'where'. > That's solved just by making names distinct. > The problem is that normal function definitions don't nest; > while you can do > > area_of_circles(Cs) -> > Area = fun (C) -> math:pi()*math:pow(C#Circle.radius, 2) end, > lists:map(Area, Cs). > > you still have to use a 'fun' to do it. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony.arcieri@REDACTED Fri Jul 22 01:58:24 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 16:58:24 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > And YEEK! The argument list here looks like nothing else in Erlang whatever! > Surely surely surely Erlang should look like Erlang! > > lists:map([1,2,3,4,5]) do (N) -> N*2 end > The "end" keyword, in all other cases in Erlang, is paired with a corresponding keyword, like "if", or "case", or "receive", or "try". Only in funs is "end" paired with the stabby symbol. If anything Erlang's fun syntax is inconsistent. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony.arcieri@REDACTED Fri Jul 22 02:17:35 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 17:17:35 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > > And YEEK! The argument list here > looks like nothing else in Erlang whatever! > Surely surely surely Erlang should look like Erlang! > > lists:map([1,2,3,4,5]) do (N) -> N*2 end > And to address your point a little more here, no I don't think "|" isn't Erlangy anymore than I feel it isn't Ruby-like. The "|" ... "|" syntax in Ruby looks like nothing else in Ruby whatsoever, but it's still aesthetically pleasing. Certainly much more so than "(" ... ")" "->" ... "end" Nor do I feel the stabby is necessary here any more than I feel the "def" token is needed when defining an anonymous function in Ruby. The "do" token alone signifies you're defining an anonymous function and matches to the corresponding "end" token. I don't believe it should be any problem to add Erlang guard syntax into the grammar as-is, I just omitted it because I knew this patch is admittedly a bit of a joke. This is where the margin turns negative. > What has mapping to do with "DO"? The do is what the mapping does. The "do" keyword is completely intuitive to me in that what follows is a function which does something. I suppose the only cognitive dissonance here is that functional programmers go out of their way to not think of their programs as actually doing something, which may make sense in a lazy language like Haskell where your program may indeed never actually do anything, but if you feel that way about a language with strict evaluation like Erlang you're just kidding yourself. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Fri Jul 22 02:25:21 2011 From: freza@REDACTED (Jachym Holecek) Date: Fri, 22 Jul 2011 01:25:21 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <20110722002521.GA5397@hanele.lan> # Tony Arcieri 2011-07-22: > On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > > And YEEK! ?The argument list here > > looks like nothing else in Erlang whatever! > Surely surely surely Erlang should look like Erlang! > > ? ?lists:map([1,2,3,4,5]) do (N) -> N*2 end > > > The "end" keyword, in all other cases in Erlang, is paired with a corresponding keyword, like "if", > or "case", or "receive", or "try". Please define what kind of correspondence you have in mind, and if you're going to mention presence of some symbols in Latin alphabet please explain why that matters at all. To me they're just syntactic items. I don't care what characters they're made of as long as those are on my keyboard and the result is easy to read and write (by me). > Only in funs is "end" paired with the stabby symbol. Let's have a look: fun (X) -> X end ^^^ ^^^ One more look: fun (X) -> X; (Y) -> Y end ^^^ ^^^ See? Happy now? :-) Have fun, -- Jachym From tony.arcieri@REDACTED Fri Jul 22 03:01:35 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 18:01:35 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <20110722002521.GA5397@hanele.lan> References: <20110722002521.GA5397@hanele.lan> Message-ID: On Thu, Jul 21, 2011 at 5:25 PM, Jachym Holecek wrote: > Please define what kind of correspondence you have in mind, and if you're > going to mention presence of some symbols in Latin alphabet please explain > why that matters at all. I mean honestly, are you trolling me? Tokens like "do" and "end" are words, or more specifically keywords. Tokens like "->" are symbols. Believe it or not for most people groupings of letters, as opposed to symbolic characters, actually has a significant meaning. To me they're just syntactic items. Can you think like a human instead of a lexer? Let's have a look: > > fun (X) -> X end > ^^^ ^^^ > > One more look: > > fun (X) -> X; (Y) -> Y end > ^^^ ^^^ > > See? Happy now? :-) No. Clause bodies are delimited by "->" and "end" with ";" as an optional clause separator. The "fun" keyword is not the delimiter and can be used in contexts where a fun has no clause body, such as fun module:function/arity. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Fri Jul 22 03:17:30 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 21 Jul 2011 20:17:30 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: So I'm messing around with using nginx's xsendfile. I have the proxy setup and its working fine. And I have the following in the config to serve the static files: location /files { root /var/www; internal; } The proxying works fine and when I make a request to the nginx server it properly ends up going to the Webmachine server, which dispatch rule sends it to my resource for static files. Here I return: NewReqData = wrq:set_resp_header("X-Accel-Redirect", "/files/test.html", ReqData), {"", NewReqData, Ctx}. But this does not work to end up having nginx serve up test.html from /var/www. Should it? Am I doing it completely wrong? Any suggestions? Thanks, Tristan On Wed, Jul 20, 2011 at 6:51 PM, Kenny Stone wrote: > One way to look at it is that Erlang has (more powerful) versions of > redis/memcached built in with ETS and Mnesia. These are pretty nice > features for web development, and I'm always a fan of cutting down the > external dependencies for deployment and dev. Just rebar your repo and you > get this powerful, self-contained web server executable (one of the selling > points of couchdb, actually). > > Kenny > > > On Wed, Jul 20, 2011 at 6:38 PM, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> Kenny, yeah, thats what I was thinking of doing as a cache method if I >> couldn't use something easily in front of Webmachine like Varnish (didn't >> actually think of Varnish until Jesper brought it up. Maybe a bit of both... >> Since while I don't want to use any templating on the backend, I want to end >> up with a general web framework from all this. >> >> Tristan >> >> >> On Wed, Jul 20, 2011 at 3:47 PM, Kenny Stone wrote: >> >>> I wonder if an ets solution wouldn't be as good, as these solutions are >>> just going to find some way of holding the data in memory anyways. WIth >>> ets, you can do things like hold compiled erlydtl/mustache templates inside >>> of it... >>> >>> Kenny >>> >>> >>> On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: >>> >>>> If you're already using Nginx and just want control over URLs, you >>>> have access to the standard rewrite module: >>>> >>>> http://wiki.nginx.org/HttpRewriteModule >>>> >>>> This is not 30x redirection btw, unless of course you want that. >>>> >>>> On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter >>>> wrote: >>>> > Sam, Jesper both these sound great, thanks! I'm also going to look >>>> into what >>>> > Jack was saying about how Rails handles some stuff. >>>> > But I'm leaning towards Jesper's idea with Varnish being the best... >>>> I'm one >>>> > of those people who scoff at most benchmarks so not sure I'll bother >>>> to do >>>> > one for this, but maybe, if someone here can A) suggest the best setup >>>> for >>>> > it B) if it makes sense at all or would just be another worthless >>>> benchmark >>>> > that really gives no information about reality. >>>> > Thanks! >>>> > Tristan >>>> > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen >>>> > wrote: >>>> >> >>>> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >>>> >> wrote: >>>> >> >>>> >> > Can anyone think of a way I can keep the nice URLs and serve the >>>> static >>>> >> > html >>>> >> > files through nginx or another webserver. >>>> >> >>>> >> Put a Varnish accelerator in front of your system >>>> >> (https://www.varnish-cache.org/). That way, it doesn't matter if >>>> your >>>> >> backend is slow at serving files as the accelerator will just cache >>>> >> static stuff for you. In addition, you avoid the trouble of going >>>> >> through another system as a proxy for static content. Also, the >>>> >> solution is quite modular. On the development system, you don't need >>>> >> more than a single system running Erlang. >>>> >> >>>> >> In my opinion, there is little reason not to plug into the whole >>>> >> industry there is where the main point is to make serving HTTP go >>>> >> faster. Trying to beat that with Erlang is probably possible, but I >>>> >> don't think it is beneficial. Varnish is really really hard to beat. >>>> >> It is built specifically for being insanely fast and it serves its >>>> >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >>>> >> blob of nasty C code. I'd rather stand on the shoulders here than >>>> >> trying to mess with it myself. >>>> >> >>>> >> >>>> >> -- >>>> >> J. >>>> > >>>> > >>>> > _______________________________________________ >>>> > erlang-questions mailing list >>>> > erlang-questions@REDACTED >>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>> > >>>> > >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Fri Jul 22 03:41:00 2011 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 21 Jul 2011 21:41:00 -0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> Message-ID: <20110722014100.GB9340@hijacked.us> On Thu, Jul 21, 2011 at 10:52:11PM +0100, Tim Watson wrote: > I'll take a closer look - sounds interesting. What put me off > originally is that there is a heck of a lot of code there, which tends > to mean there's a lot going on whereas I wanted minimal impact and > control over blocking the client (or better, making the work in the > client) or not. Fastlog is only 665 lines of code, as it does very > little work to get from the call site to the underlying logging module > (error_logger by default). Unfortunately, error_logger has a lot of flaws, which is why lager has so much code to replace error_logger in it. Andrew From octopusfluff@REDACTED Fri Jul 22 04:02:24 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Thu, 21 Jul 2011 19:02:24 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <20110722002521.GA5397@hanele.lan> Message-ID: On Thu, Jul 21, 2011 at 6:01 PM, Tony Arcieri wrote: >> To me they're just syntactic items. > > Can you think like a human instead of a lexer? This appears a little antagonistic. Jachym's responses appeared to be in good faith and serious to my perspective. I also don't make any kind of distinction between keywords with letters and keywords that are symbols, and I'm not clear on why this is important to change, at least in isolation. As Richard pointed out, if you're going to make syntax cleaner, you need to do it comprehensively. And I personally don't see any readability improvement, nor anything this permits us to do we couldn't before. From tony.arcieri@REDACTED Fri Jul 22 04:19:50 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 19:19:50 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <20110722002521.GA5397@hanele.lan> Message-ID: On Thu, Jul 21, 2011 at 7:02 PM, Amy Lear wrote: > This appears a little antagonistic. Jachym's responses appeared to be > in good faith and serious to my perspective. > > I also don't make any kind of distinction between keywords with > letters and keywords that are symbols, and I'm not clear on why this > is important to change, at least in isolation. As Richard pointed out, > if you're going to make syntax cleaner, you need to do it > comprehensively. > > And I personally don't see any readability improvement, nor anything > this permits us to do we couldn't before. > Well, my apologies, but I'm having a bit of trouble wrapping my brain around the lack of aesthetic sense here. So forgive me for overreacting. Here are some strategies to delimiting tokens that make sense to me: "{" ... "}" - the oft reviled curly brace approach. These are a paired set of tokens that match. "beginningtoken" ... "endingtoken" - these match at least in that they're words, and perhaps their meaning describes how they represent the beginning and end of a block of code "->" ... "." - these don't do a great job of matching up but they're kind of acceptable because they're both symbols. The "->" character looks like an arrow and makes a decent enough beginning token. The "." token has a long history as the period character at the end of a sentence and imparts this is where a function ends. Now compare this to: "->" ... "end" Seriously. WTF is that? Those two tokens do not match up whatsoever. The "->" token, elsewhere in Erlang, is found in "forms", which are statements in that they do not return a value. They define functions and can't be used from things which only comprehend Erlang expressions, such as eshell. The "end" token, elsewhere in Erlang, is only found in expressions, like "case", "if", "receive", and "try". Expressions return a value. Erlang fun syntax jams together the "->" token and the "end" token in Erlang's lambda expression. In my opinion "->" does not belong in this context. All of the other Erlang expressions are delimited by keywords. "->" and "end" just don't match up as beginning and ending tokens. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Fri Jul 22 04:28:35 2011 From: freza@REDACTED (Jachym Holecek) Date: Fri, 22 Jul 2011 03:28:35 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <20110722002521.GA5397@hanele.lan> Message-ID: <20110722022835.GA7840@hanele.lan> # Tony Arcieri 2011-07-22: > On Thu, Jul 21, 2011 at 5:25 PM, Jachym Holecek wrote: > > Please define what kind of correspondence you have in mind, and if you're > going to mention presence of some symbols in Latin alphabet please explain > why that matters at all. > > I mean honestly, are you trolling me? No -- there was an attempt at humor later on, but overall I'm serious. > Tokens like "do" and "end" are words, or more specifically keywords. Tokens like "->" are symbols. > Believe it or not for most people groupings of letters, as opposed to symbolic characters, actually > has a significant meaning. Are you sure you'd convince a Chinese programmer with little to no English skills here? I remember that when I was starting programming the language had an UNLESS statement (some dialect of BASIC I think, too tired to be sure ATM) -- didn't know what that word meant, but that hardly stopped me... Sure, it was nice that it was in Latin alphabet instead of Greek or something, but it (= it's word-y nature) wasn't terribly significant either. > Let's have a look: > > ?fun (X) -> X end > ?^^^ ? ? ? ? ?^^^ > > One more look: > > ?fun (X) -> X; (Y) -> Y end > ?^^^ ? ? ? ? ? ? ? ? ? ?^^^ > > See? Happy now? :-) > > No. Clause bodies are delimited by "->" and "end" with ";" as an optional clause separator. The > "fun" keyword is not the delimiter and can be used in contexts where a fun has no clause body, such > as fun module:function/arity. Wait. This was in response to: The "end" keyword, in all other cases in Erlang, is paired with a corresponding keyword, like "if", or "case", or "receive", or "try". Only in funs is "end" paired with the stabby symbol. If anything Erlang's fun syntax is inconsistent. Which is wrong like I say, to quote lib/stdlib/src/erl_parse.yrl: fun_expr -> 'fun' fun_clauses 'end' : build_fun(?line('$1'), '$2'). fun_clauses -> fun_clause : ['$1']. fun_clauses -> fun_clause ';' fun_clauses : ['$1' | '$3']. The "fun M:F/A" is an atomic thing, no need for pairing there as there's no expression to enclose. And "->" has nothing to do with ";" -- the former separates pattern from clause body, the latter separates clauses from one another. If you're really talking just about the aesthetical aspects of code seen as sequence of characters (regardless of the abstract structure it encodes) than well, you're surely free to, I just don't see the point -- and actually it's possibly counterproductive too: it's this clear underlying structure that (when done right) makes code feel intuitive and easy to work with. For example, in your proposal, can I pattern-match on arguments? Use guards? If so, will they look the same they usually look? Can I do multi-clause "do"s? How comes you're depriving functions of bits of their arity (this kind of implicit trickery doesn't happen anywhere else in Erlang)? Is this proposal limited to cases that have a Fun as first argument? That again is a new thing in the language. So: why replace perceived inconsistency in the language by actual hardcore one (seems that way so far)? More importantly: aren't there more significant aspects of the language to focus on instead of something that's already handled pretty well? (Sure, your call, just saying) BR, -- Jachym From ok@REDACTED Fri Jul 22 04:39:17 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 14:39:17 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <4013714A-5AA6-4492-98AA-7C1C422A8B73@cs.otago.ac.nz> On 22/07/2011, at 11:34 AM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > > However, I'd rather ask: can Erlang have something like Ruby like > > blocks? Yes, yes it can. > > Well yes, it does. They are called funs. > > This sort of sentiment lacks any sense of aesthetics. On the contrary, it is motivated by a very strong sense of fitness. > It is akin to saying that a potato sack is the same thing as a suit coat because both provide the same basic function of covering your torso. No, it's like saying that a family sedan is pretty much the same thing as a sports car with lots of extra chromium, several more wheels spinning idly, and a red plastic nose on the front, except that the family sedan is more useful. It has been pointed out several times in this thread that funs (or Smalltalk-style blocks) can be used easily in far more contexts than Ruby "blocks". > Let me call out explicitly what is ugly about Erlang fun syntax: it combines a symbolic token "->" with a keyword token "end" instead of a matching pair of symbolic tokens (e.g. "{" ... "}" or even "->" ... ".") or a matching pair of keywords (e.g. "do" ... "end"). The phrase "symbolic token" as used here is new to me. One reason I don't like that phrase is that "->" "{" and "}" are precisely *NOT* symbolic in any useful sense. They are just punctuation. In PL/I, x->y means "follow untyped pointer x to typed field y". In C, x->y means "follow typed pointer x to typed field y". In Strudel, x->y->z means "add an edge from x to z labelled y". In Dot, x->y means "add an edge from x to y". In logic, x->y means "y or not x". In Pop-2 and S, x->y means "assign the value of y to the variable x". In a Scheme library I have, (-> x y z...) means "send message y to the value of x with the values of z... as arguments." In Prolog, P->Q;R means "if P then Q else R". I could go on. And don't get me started on possible meanings of { and }. (Take a look at J...) You can *call* the punctuation marks "symbolic tokens" all you want, the problem is that they aren't symbolic *OF* anything in particular, except by arbitrary convention in a specific system of notation. By the way, Erlang funs ***DON'T*** "combine a symbolic token with a keyword token". They use two keywords for brackets. The opening bracket is 'fun'. The closing bracket is 'end'. The arrow is NOT a bracket. You say it would be better to use "a matching pair of keywords"? Well, that's *EXACTLY* what Erlang does. Funs use "->" to separate argument patterns and guards from bodies. This is *consistent* throughout the whole of Erlang: top level functions, if, case, receive, ... EVERYTHING uses "->" to separate patterns-and-guards from bodies. If you want to separate some patterns-and-guards from a body, you use "->". Nothing else. If you see a "->", it is separating some patterns-and-guards from a body. Nothing else. One of the things that is horrible about the proposal to add funs-crippled-in-the-Ruby-manner to Erlang is that it BREAKS THIS CONSISTENCY. foobar do |X Y| end *FAILS* to - enclose the arguments in parentheses, like ordinary functions - separate the arguments-and-guards from the body with "->", like ordinary functions. This makes the language MORE complicated LESS consistent for no real gain. > I think there are a lot of people in the Erlang community who are either completely oblivious to how that sort of thing harms the aesthetics of the language or willfully choose to ignore it. This makes the Erlang "fun" syntax awkward and clumsy and not particularly "fun", when really anonymous functions are a powerful concept and should be a pleasure to use. Smalltalk: [ :x :y | some expression involving x and y] Erlang: fun (X, Y) -> some expression involving X and Y end Erlang uses matching keywords 'fun' 'end' instead of '[' ']'. Erlang uses '(' and ',' instead of ':'. Erlang uses ')' where Smalltalk has nothing, but then Erlang has pattern matching to fit in here. Erlang uses '->' where Smalltalk has '|'. OK, suppose we take advantage of Unicode, and suppose we lose the argument parentheses. fun (X, Y) -> e(X, Y) end => ?X, Y ? e(X, Y)? We could do that easily, while still allowing the old syntax. In a major revision of Erlang syntax, we should certainly give serious thought to taking advantage of Unicode (without requiring it). But just how much of a change is this, really? Would that be enough to satisfy the Aubrey Beardsleys in the Erlang mailing list? I've said it before: Erlang syntax *is* clunky, but what merits reconsideration (which might well ratify the existing syntax as a workable compromise) is the *whole* language, not this part or that part. This is a textbook example of a change that would make the language WORSE by adding fragile support for a special case in a way that makes the language *AS A WHOLE* less consistent, without adding ANY extra power, and very little in the way of convenience. As for me, I find fun ... end *clunky* but *tasteful*. I know it could be done with fewer characters in other languages, but it has been done in a way that is coherent with the rest of the language, and such coherence is an aesthetic (and mnemonic!) virtue not to be sacrificed lightly. just one thing. We can add new syntax for things we could not do easily before, but From ok@REDACTED Fri Jul 22 04:41:32 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 14:41:32 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: I just _had_ to include this quotation from today's "Haskell Weekly Newsletter": PenguinOfDoom: Being enlightened gentlemen, we split all programming languages into two groups, sucks and doesn't-suck and put all of them into the first group. From ok@REDACTED Fri Jul 22 04:43:11 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 14:43:11 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> On 22/07/2011, at 11:58 AM, Tony Arcieri wrote: > > The "end" keyword, in all other cases in Erlang, is paired with a corresponding keyword, like "if", or "case", or "receive", or "try". Only in funs is "end" paired with the stabby symbol. Wrong. In funs, "end" is paired with "fun" and "fun" ONLY. From watson.timothy@REDACTED Fri Jul 22 04:49:25 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 03:49:25 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: <20110722014100.GB9340@hijacked.us> References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: On 22 July 2011 02:41, Andrew Thompson wrote: > On Thu, Jul 21, 2011 at 10:52:11PM +0100, Tim Watson wrote: >> I'll take a closer look - sounds interesting. What put me off >> originally is that there is a heck of a lot of code there, which tends >> to mean there's a lot going on whereas I wanted minimal impact and >> control over blocking the client (or better, making the work in the >> client) or not. Fastlog is only 665 lines of code, as it does very >> little work to get from the call site to the underlying logging module >> (error_logger by default). > > Unfortunately, error_logger has a lot of flaws, which is why lager has > so much code to replace error_logger in it. > > Andrew You're not the first person to mention this, so clearly there are compelling reasons to do so. My motivation for sticking with error_logger was that it has been battle tested for a very long time, and is therefore reliable. Clearly there are times when it doesn't do what you want (hence the appearance of riak_err and other customisations) so a replacement is a good idea. If Lager is going to be what riak uses for its error logging, I suspect that'll persuade a lot of people that it's a worthwhile alternative. My primary motivation for building fastlog was to get control over the logging levels (at runtime), which is does fine for now. I will be keeping an eye on Lager though - as it does sound very capable. Good luck with it! From ok@REDACTED Fri Jul 22 04:54:51 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 14:54:51 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <35955507-E415-457E-8FC1-7EA842F30F13@cs.otago.ac.nz> On 22/07/2011, at 12:17 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 3:48 PM, Richard O'Keefe wrote: > > And YEEK! The argument list here > looks like nothing else in Erlang whatever! > Surely surely surely Erlang should look like Erlang! > > lists:map([1,2,3,4,5]) do (N) -> N*2 end > > And to address your point a little more here, no I don't think "|" isn't Erlangy anymore than I feel it isn't Ruby-like. The "|" ... "|" syntax in Ruby looks like nothing else in Ruby whatsoever, but it's still aesthetically pleasing. Certainly much more so than "(" ... ")" "->" ... "end" The "|" ... "|" notation really does NOT mesh very well with pattern matching, something Ruby is, um, deficient in. Overloading punctuation marks with too many meanings is not a good idea in any language. > > Nor do I feel the stabby Why do you call the right arrow "stabby"? To me that > is necessary here any more than I feel the "def" token is needed when defining an anonymous function in Ruby. The "do" token alone signifies you're defining an anonymous function and matches to the corresponding "end" token. Just *precisely* as the "fun" token alone signifies that you are defining an anonymous function and matches the corresponding "end" token in Prolog. *Some* token is needed to separate the arguments from the body. Since the arrow is used for that everywhere else in Erlang, it would be inconsistent not to use it here. > =This is where the margin turns negative. > What has mapping to do with "DO"? > > The do is what the mapping does. No. "do" is a VERB. Mapping is about computing *values*. Mapping is not about side effects. "DO" is about as imperative a keyword as you could expect to find; it really does not belong in a mostly functional language. If you want to talk about aesthetics, "do" has got to go. Smalltalk gets this right: container do: aBlock DOES something for each element. container collect: aBlock COLLECTS a value for each element (it's map). container select: aBlock SELECTS elements (it's filter) container count: aBlock COUNTS matching elements container sortedBy: aBlock SORTS container BY a comparison block Ruby seized on one special case, "do:", emptied it of all meaning, and shifted the burden of saying what result to compute elsewhere. Not an improvement. > The "do" keyword is completely intuitive to me in that what follows is a function which does something. I suppose the only cognitive dissonance here is that functional programmers go out of their way to not think of their programs as actually doing something, which may make sense in a lazy language like Haskell where your program may indeed never actually do anything, but if you feel that way about a language with strict evaluation like Erlang you're just kidding yourself. What has strict evaluation to do with it? It's perfectly possible to have a pure functional language (that is, no mutable data structures and no side effects anywhere in the semantics) that is strict. It is also possible to have a lazy language that *does* have side effects, see S for an example. Purity and laziness are logically and practically independent. To imply *EVERY* time that a function works by side effects is no kindness. From ok@REDACTED Fri Jul 22 05:10:24 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 15:10:24 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <20110722002521.GA5397@hanele.lan> Message-ID: <6D35E385-DCBF-445D-9DE3-54BBADE43C57@cs.otago.ac.nz> On 22/07/2011, at 1:01 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 5:25 PM, Jachym Holecek wrote: > Please define what kind of correspondence you have in mind, and if you're > going to mention presence of some symbols in Latin alphabet please explain > why that matters at all. > > I mean honestly, are you trolling me? No, he pretty obviously wasn't. > > Tokens like "do" and "end" are words, or more specifically keywords. And in Algol 60, which pretty much invented the concept, keywords counted as single characters in the abstract alphabet and could be freely replaced by other words or even actual characters in implementations. (This also applied in Algol 68.) > Tokens like "->" are symbols. Believe it or not for most people groupings of letters, as opposed to symbolic characters, actually has a significant meaning. OK, so what's the significant meaning of An bhfuil an fear tromch?iseach? without looking it up? > > To me they're just syntactic items. > > Can you think like a human instead of a lexer? I don't know Jachym Holecek at all, but with that name, I'm guessing he might be from the Czech Republic. We might all, from time to time, try thinking like a human being whose native language is not English and isn't particularly English-like. To such a person, a keyword like SML's "fn" or Erlang's "bsl" has no more significant meaning than Haskell's "\" or C's "<<". > > Let's have a look: > > fun (X) -> X end > ^^^ ^^^ > > One more look: > > fun (X) -> X; (Y) -> Y end > ^^^ ^^^ > > See? Happy now? :-) > > No. Clause bodies are delimited by "->" and "end" with ";" as an optional clause separator. The "fun" keyword is not the delimiter and can be used in contexts where a fun has no clause body, such as fun module:function/arity. This is so wrong an understanding of Erlang syntax that I don't know where to start. Consider this: _____________ ______________ fun (true) -> f() ; (false) -> g() end | |_____ |_____ | |____________________________________| The "fun" matches the "end". NEITHER "->" matches the "end". Neither "->" has ANY matching token at the right. If the rightmost "->" matched the "end", what would the leftmost one be doing? If the leftmost "->" matched the "end", what kind of thing would (body ; arguments -> body) be? You need an understanding of Erlang syntax that works with legal Erlang examples. It's lambda : FUN fun_clauses END | FUN fun_name ; fun_clauses : fun_clause | fun_clauses SEMI fun_clause ; fun_clause : fun_head ARROW body ; fun_head : arguments WHEN guard | arguments ; The fact that "fun" can be used without arguments or body or "end" can in no way be taken to mean that it is not the delimiter when there _is_ an end, just as the fact that "-" can be used without a left operand means that it isn't an infix operator when it _does_ have a left operand. In this case, ";" and "->" *look* like and *act* like infix operators. Neither of them is a bracket, here or anywhere else. From tony.arcieri@REDACTED Fri Jul 22 05:49:07 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 20:49:07 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 7:43 PM, Richard O'Keefe wrote: > Wrong. In funs, "end" is paired with "fun" and "fun" ONLY. > Okay, conceded (and Jachym made the same point) I'll revert to my other point, which is that the '->' token in Erlang is only found in forms outside of funs, which are expressions. No other expression in Erlang uses the '->' token. I'd also ask what you feel about the people who responded to this thread who find the Erlang fun syntax awkward. I certainly find it awkward. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 22 05:35:07 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 15:35:07 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <20110722002521.GA5397@hanele.lan> Message-ID: <1E8430AE-9F0A-4F3C-ABBB-F3F7FB2A0618@cs.otago.ac.nz> On 22/07/2011, at 2:19 PM, Tony Arcieri wrote: > Here are some strategies to delimiting tokens that make sense to me: > > "{" ... "}" - the oft reviled curly brace approach. These are a paired set of tokens that match. Who reviles it? Nobody in Erlang. We use those for tuples. (And SETL uses them for sets, and Smalltalk uses them for arrays, and ...) > > "beginningtoken" ... "endingtoken" - these match at least in that they're words, and perhaps their meaning describes how they represent the beginning and end of a block of code Hey, I'm an Ada fan. > > "->" ... "." - these don't do a great job of matching up AND IN ERLANG NOBODY EVER DREAMED THAT THEY MIGHT BE THOUGHT OF AS MATCHING UP. They do not. Consider max(X, Y) when X >= Y -> X; max(X, Y) when X < Y -> Y. Think of ";" as an infix operator: ?max(X, Y) when X >= Y -> X? ; ?max(X, Y) when X < Y -> Y?. I've put the corner brackets in to show clearly what the "operands" of ";" are. You will notice that the "->" tokens are INSIDE the corner brackets. There is no sense whatever in which they can be said to match the ".". The "." is outside the corner brackets. NO token matches it. Now think of "when" and "->" as infix operators. ???max(X, Y)? when ?X >= Y?? -> ?X?? ; ???max(X, Y)? when ?X < Y??-> ?Y??. "when", "->", and ";" look like infix operators and in many ways act like infix operators. They can certainly be parsed as infix operators. NONE of these three tokens "matches" any other token, EVER. Not "." and not "end". One hypothesis occurs to me, and that is that you have formed a mental model of Erlang based on functions and funs with only one clause, where you might see a parallel between [](int x, int y) { return x + y; } // C++ ^(int x, int y) { return x + y; } // Objective C fun ( X, Y) -> X + Y end and think that "fun" is like "[]" or "^", "->" is like "{", and "end" (or ".") is like "}". But that is a false analogy which breaks down horribly when you have multiple clauses. If it's not your analogy, my apologies, but it is the only way I can imagine someone coming up with the crazy idea that "->" *ever* matches "end" or ".". > Now compare this to: > > "->" ... "end" > > Seriously. WTF is that? It is a radical misunderstanding of Erlang syntax, that's what it is. There is *NO* syntactic unit in Erlang *anywhere* that has the form "->" ... "end". None. > Those two tokens do not match up whatsoever. > > The "->" token, elsewhere in Erlang, is found in "forms", which are statements in that they do not return a value. Wrong. There are no "forms" in Erlang which do not return values. "if", "case", "receive", and "try" are *ALL* of them *expressions* with well defined values. It's true that in C/C++/ObjC/JavaScript/Java/C# "if" and "case" and "try" are statements. So much the worse for them. Erlang is not any of those languages. (If you want to compare it with any imperative language, try Algol 68, in which also "if" and "case" were expressions, not statements.) Come to think of it, according to http://www.ruby-doc.org/docs/ProgrammingRuby/ the "if" construct in Ruby is also an expression, it returns a value, and so does "case". > They define functions and can't be used from things which only comprehend Erlang expressions, such as eshell. > > The "end" token, elsewhere in Erlang, is only found in expressions, like "case", "if", "receive", and "try". Expressions return a value. "funs" are no exception. The pattern that is common to all of them is '->' {';' '->' }... 'end' There is nothing whatsoever exceptional about funs in this regard. From tony.arcieri@REDACTED Fri Jul 22 06:10:49 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 21:10:49 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: All right so yeah, I take this back too. I guess the real way of looking at it is: fun [clauses] end And if you look at it like that, funs are consistent with other expressions. Funs are not consistent with other forms. Mea culpa. Perhaps I could re-evaluate this patch with a different syntax, one that works like: function(...) do [clauses] end So apologies there Mr. Richard O'Keefe. You had it right and I was mistaken. On Thu, Jul 21, 2011 at 8:49 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 7:43 PM, Richard O'Keefe wrote: > >> Wrong. In funs, "end" is paired with "fun" and "fun" ONLY. >> > > Okay, conceded (and Jachym made the same point) > > I'll revert to my other point, which is that the '->' token in Erlang is > only found in forms outside of funs, which are expressions. No other > expression in Erlang uses the '->' token. > > I'd also ask what you feel about the people who responded to this thread > who find the Erlang fun syntax awkward. I certainly find it awkward. > > -- > Tony Arcieri > > -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Fri Jul 22 06:12:52 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Thu, 21 Jul 2011 21:12:52 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 8:49 PM, Tony Arcieri wrote: > I'd also ask what you feel about the people who responded to this thread who > find the Erlang fun syntax awkward. I certainly find it awkward. Sure, it's awkward. So are records. There's awkward stuff in lots of languages. The key here is, from my (and possibly others') perspective your suggestion isn't less awkward, or more readable, or enable constructs that were previously difficult, cumbersome, or impossible. When you re-approach it, try to determine what exactly you gain for the effort of changing a well-established syntax. If it doesn't actually let anyone do anything new, it's likely to meet quite a bit of resistance. From tony.arcieri@REDACTED Fri Jul 22 06:15:03 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 21:15:03 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 9:12 PM, Amy Lear wrote: > Sure, it's awkward. So are records. > The difference is I think Richard O'Keefe feels my pain on the issue of records. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From octopusfluff@REDACTED Fri Jul 22 06:18:30 2011 From: octopusfluff@REDACTED (Amy Lear) Date: Thu, 21 Jul 2011 21:18:30 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 9:15 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 9:12 PM, Amy Lear wrote: >> >> Sure, it's awkward. So are records. > > The difference is I think Richard O'Keefe feels my pain on the issue of > records. EVERYONE feels the pain on the issue of records. From max.lapshin@REDACTED Fri Jul 22 06:22:56 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Jul 2011 08:22:56 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: > You're not the first person to mention this, so clearly there are > compelling reasons to do so. My motivation for sticking with > error_logger was that it has been battle tested for a very long time, > and is therefore reliable. It has been tested for several times and considered a VM OOM killer, when you have to deal more than with 1 MB of state. This is why it is one of the first things, done in erlyvideo: turn off error_logger Erlang is a platform is excelent. erlyvideo can crush down only because of error_logger and it happened earlier. From tony.arcieri@REDACTED Fri Jul 22 06:26:57 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 21:26:57 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 9:18 PM, Amy Lear wrote: > EVERYONE feels the pain on the issue of records. > Except Scumbag Ericsson: http://i.imgur.com/ScMJX.jpg -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 22 06:28:02 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 16:28:02 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> On 22/07/2011, at 3:49 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 7:43 PM, Richard O'Keefe wrote: > Wrong. In funs, "end" is paired with "fun" and "fun" ONLY. > > Okay, conceded (and Jachym made the same point) > > I'll revert to my other point, which is that the '->' token in Erlang is only found in forms outside of funs, which are expressions. No other expression in Erlang uses the '->' token. I am totally confused here. Erlang constructs "->" if and only if they have patterns-and-guards (well, patterns-and-or-guards, to be picky) to separate from bodies. That's it. No exceptions. I have just revised the Expressions chapter of the reference manual, and I cannot find one single example of a kind of expression that _should_ use "->" but doesn't. So what kind of expression do _you_ think ought to use "->" but doesn't? > I'd also ask what you feel about the people who responded to this thread > who find the Erlang fun syntax awkward. I certainly find it awkward. Oh dear, this is going to sound insulting, but it's not meant that way. I am more impressed by people who demonstrate that they understand how the current syntax actually *works* before they complain about it. I myself have said that I find Erlang syntax "clunky but tasteful". It uses more tokens than Haskell does. But then, Haskell syntax is surprisingly tricky, and I'm not sure how many Haskell parsers actually get it 100% right. For reference, it took a huge amount of fighting in the ISO Prolog committee to agree on an unambiguous syntax, and in the revision mailing list there are continuing, um, strong disagreements. I think the restrictions compared with classic Prolog were and are too severe. But a correspondent who has done a heck of a lot more work on standards issues in recent years than I have points out with a great deal of justice that the various Prolog implementations out there STILL don't agree on syntax, not as much as their authors fondly imagine. He'd like to see them get what's in the standard right before adding anything else. Do not despise a simple syntax that implementors can actually get RIGHT. It is *HARD* to come up with a clean coherent syntax for the whole of a programming language. And it is triply hard to patch your way there, and outright impossible to do it by adding inconsistencies. Yes, "fun ... end" is not as pretty as it could be, but for something that was added years after Erlang was shipping, it's nearly as close to the rest of the language as anyone could wish for. (The thing I don't like is the scope rules for variables shared between "fun" heads and their contexts, which has nothing to do with keywords or punctuation.) If you don't like funs in Erlang, try Objective C or JavaScript for a while. From tony.arcieri@REDACTED Fri Jul 22 06:29:32 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 21:29:32 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 9:28 PM, Richard O'Keefe wrote: > It is *HARD* to come up with a clean coherent syntax for the whole of a > programming language. Dude, I know! I mean, I tried to make my own language and all. Maybe I'll try again. But it won't be on the Erlang VM. -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 22 06:30:01 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 16:30:01 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> Message-ID: <154C9756-0BD4-41AA-BBF3-232792675003@cs.otago.ac.nz> On 22/07/2011, at 4:26 PM, Tony Arcieri wrote: > On Thu, Jul 21, 2011 at 9:18 PM, Amy Lear wrote: > EVERYONE feels the pain on the issue of records. > > Except Scumbag Ericsson: You pay them how much for support? From tony.arcieri@REDACTED Fri Jul 22 06:38:16 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Thu, 21 Jul 2011 21:38:16 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <154C9756-0BD4-41AA-BBF3-232792675003@cs.otago.ac.nz> References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> <154C9756-0BD4-41AA-BBF3-232792675003@cs.otago.ac.nz> Message-ID: On Thu, Jul 21, 2011 at 9:28 PM, Richard O'Keefe wrote: > I am more impressed by people who demonstrate that they understand how > the current syntax actually *works* before they complain about it. > My complaint came in the form of a patch to Erlang's yecc grammar. I think that demonstrates a degree of understanding that most Erlang programmers don't have. I admit I made mistakes in some of my follow-up responses because I wasn't referencing the grammar at the time I made the response. Mea culpa. Did I mention I implemented an entire language to the point of borderline usability on the Erlang VM? > On Thu, Jul 21, 2011 at 9:18 PM, Amy Lear wrote: > > EVERYONE feels the pain on the issue of records. > > > > Except Scumbag Ericsson: > > You pay them how much for support? LOL okay dude. Are you happy with the rejection of your records proposals? -- Tony Arcieri -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 22 07:37:56 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 22 Jul 2011 17:37:56 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> <154C9756-0BD4-41AA-BBF3-232792675003@cs.otago.ac.nz> Message-ID: <666CE2B7-2465-4470-A98F-7755BF96B106@cs.otago.ac.nz> On 22/07/2011, at 4:38 PM, Tony Arcieri wrote: > > LOL okay dude. Are you happy with the rejection of your records proposals? No. Any more than I imagine Joe Armstrong is happy that his proposal for pretty much the same thing has come to nothing. Actually, there is no *rejection*. It's just that nobody has got around to deciding anything either way yet. For Erlang's audience, I think - fixing the major system integrity weakness of too many atoms blowing Erlang out of the water - a logging system that deals with all the issues recently mentioned - line numbers in exception reports - better tracing/monitoring tools are probably more urgent. There is a qualitative difference between things that are an irritation when coding and things that can bring your system down or things that help you fix systems while customers are screaming at you. I've made one proposal in the "things that can bring your system down" area, and whatever my tastes in language design, I would *hope* that that issue would treated as more urgent. (And I'm quite disappointed that it has apparently not been treated as very urgent, because it has quite a warping effect on data structure design.) If you've got a language nearly to the point of usability on the Erlang VM, presumably you understand that VM far better than I do. I never did manage to get my head around the Beam instruction set, so if you want to do something extremely helpful, writing the Beam manual that should always have existed but doesn't would be close to the top of the list. From paul.burt@REDACTED Fri Jul 22 09:39:22 2011 From: paul.burt@REDACTED (Paul Burt) Date: Fri, 22 Jul 2011 08:39:22 +0100 Subject: [erlang-questions] Problems with re:replace/4 Message-ID: Hi, I'm a newcomer to Erlang and am interested in writing some helper functions to parse and manipulate responses from the Twitter API for use in a web application. What I would like to do is replace "@mention" and "#hashtag" with HTML markup as follows: @mention --> @mention #hashtag --> #hashtag In other words transform Twitter @mentions and #hashtags into clickable links. I have the basic structure of a function (tweetify/1) to do this below. The ???? represent where I'm running into trouble: tweetify(Input) -> Replacements = [ {"@(?\\w+)", "&"}, {"#(?\\w+)", "&"} ], lists:foldl(fun({RE, Replacement}, Tweet) -> re:replace(Tweet, RE, Replacement, [global, {return, binary}]) end, Input, Replacements). Whilst I understand that the token "&" will insert _everything_ matched by the regular expression (i.e. @mention and #hashtag respectively), how do I use the named tags in the replacement string? In other words, how do I get hold of \k and \k in the PCRE idiom and use them in the replacement string? Thanks in advance for any assistance/pointers. -Paul From antoine.koener@REDACTED Fri Jul 22 09:51:08 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Fri, 22 Jul 2011 09:51:08 +0200 Subject: [erlang-questions] Problems with re:replace/4 In-Reply-To: References: Message-ID: <83E7466F-1823-41A5-81E4-1F12B71E0D80@gmail.com> Whenever you deal with re module and the \ character, try \\ instead. There's 2 levels of filtering. Maybe this will help :)) On 22 juil. 2011, at 09:39, Paul Burt wrote: > Hi, > > I'm a newcomer to Erlang and am interested in writing some helper > functions to parse and manipulate responses from the Twitter API for > use in a web application. > > What I would like to do is replace "@mention" and "#hashtag" with HTML > markup as follows: > > @mention --> @mention > #hashtag --> #hashtag > > In other words transform Twitter @mentions and #hashtags into clickable links. > > I have the basic structure of a function (tweetify/1) to do this > below. The ???? represent where I'm running into trouble: > > tweetify(Input) -> > Replacements = [ > {"@(?\\w+)", "&"}, > {"#(?\\w+)", " href=\"http://search.twitter.com/search?tag=????\">&"} > ], > lists:foldl(fun({RE, Replacement}, Tweet) -> > re:replace(Tweet, RE, Replacement, [global, > {return, binary}]) > end, Input, Replacements). > > Whilst I understand that the token "&" will insert _everything_ > matched by the regular expression (i.e. @mention and #hashtag > respectively), how do I use the named tags in the replacement string? > In other words, how do I get hold of \k and \k in > the PCRE idiom and use them in the replacement string? > > Thanks in advance for any assistance/pointers. > > -Paul > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Fri Jul 22 09:55:08 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 22 Jul 2011 09:55:08 +0200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> Message-ID: On Fri, Jul 22, 2011 at 6:28 AM, Richard O'Keefe wrote: > > On 22/07/2011, at 3:49 PM, Tony Arcieri wrote: > >> On Thu, Jul 21, 2011 at 7:43 PM, Richard O'Keefe wrote: >> Wrong. ?In funs, "end" is paired with "fun" and "fun" ONLY. >> >> Okay, conceded (and Jachym made the same point) >> >> I'll revert to my other point, which is that the '->' token in Erlang is only found in forms outside of funs, which are expressions. No other expression in Erlang uses the '->' token. > > I am totally confused here. > Erlang constructs "->" if and only if they have patterns-and-guards > (well, patterns-and-or-guards, to be picky) to separate from bodies. > > That's it. ?No exceptions. > > I have just revised the Expressions chapter of the reference manual, > and I cannot find one single example of a kind of expression that > _should_ use "->" but doesn't. > > So what kind of expression do _you_ think ought to use "->" but doesn't? > >> I'd also ask what you feel about the people who responded to this thread >> who find the Erlang fun syntax awkward. I certainly find it awkward. > > Oh dear, this is going to sound insulting, but it's not meant that way. > > I am more impressed by people who demonstrate that they understand how > the current syntax actually *works* before they complain about it. > > I myself have said that I find Erlang syntax "clunky but tasteful". > It uses more tokens than Haskell does. > > But then, Haskell syntax is surprisingly tricky, and I'm not sure > how many Haskell parsers actually get it 100% right. > > For reference, it took a huge amount of fighting in the ISO Prolog > committee to agree on an unambiguous syntax, and in the revision mailing > list there are continuing, um, strong disagreements. ?I think the > restrictions compared with classic Prolog were and are too severe. ?But > a correspondent who has done a heck of a lot more work on standards issues > in recent years than I have points out with a great deal of justice that > the various Prolog implementations out there STILL don't agree on syntax, > not as much as their authors fondly imagine. ?He'd like to see them get > what's in the standard right before adding anything else. > > Do not despise a simple syntax that implementors can actually get RIGHT. > > It is *HARD* to come up with a clean coherent syntax for the whole of a > programming language. ?And it is triply hard to patch your way there, > and outright impossible to do it by adding inconsistencies. > > Yes, "fun ... end" is not as pretty as it could be, but for something > that was added years after Erlang was shipping, it's nearly as close > to the rest of the language as anyone could wish for. ?(The thing I > don't like is the scope rules for variables shared between "fun" heads > and their contexts, which has nothing to do with keywords or > punctuation.) > > If you don't like funs in Erlang, try Objective C or JavaScript for a while. Now you've set me off ... If you think Erlang funs are problematic you should take a look at javascript. Javascript closures are truly horrible NOTE in what follows "$$" means the javascript shell prompt $$ x = 10; 10 Now let's make a function y that adds x to its argument Pretty simply, eh, In Erlang I'd write fun(Y) -> Y + X end In Javascript I write: $$ f = function(y){return(x+y);}; function (y){return(x+y);} Test it: $$ f(5); 15 Hooray - great - but wait a moment Now change x $$ x=20; 20 Now if the force was with javascript I'd rather hope that I had not accidentally broken f. What happens? Now what was f(5), let's take a look: $$ f(5); 25 Oh dear - maths is broken. f(5) is not what it was earlier Try again: $$ x = 10; 10 Add an extra level of indirection and throw in some semicolons $$ f = (function(z){return function(y){return y+z;};})(x); (( F = fun(Y) -> Y + X end, in Erlang :-)) function (y){return y+z} $$ f(5); 15 So far so good. Now lets change x $$ x=20; 20 Have we broken f? $$ f(5); 15 No f still works. No Lets' go back to Erlang and repeat the argument as it were. Let's pretent Erlang is javascript > X = 10; > F = fun(Y) -> X + Y end; Now ask the javascript question "what happens if I now change X" Erlang answer - you can't Now if you think horrible incantations like f = (function(z){return function(y){return y+z;};})(x); in Javascript solved all your problems with closures think again. What you do inside the {...} bits of a function depends crucially on the semantics of assignment in javascript. I have three javascript books at home - I looked up assignment in all three. But none of them said anything sensible ... What does x = y; mean in javascript? If I say x = y; In javascript and then change y, is x changed? Or if I change x is y changed. Is x a deep or shallow copy of y - well the answer is of course "it depends" Now in Erlang X = Y behaves as if you'd made a deep copy of Y. Of course you don't actually make a deep copy of Y - you just think you have, and the question "what happens to X if we subsequently change Y?" is easy to answer "You can't change Y" (actually - non of the three Erlang books mention this, if you come from javascript you have to worry about the semantics of assignment, and the deep and shallow copying implied by assignment) Erlang fun syntax might have the odd wart (variable bindings can sometimes seem strange to the unaccustomed eye) but at least things bound in closures are actually bound in the closure and cannot be changed later - this means no nasty surprises later. Cheers /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From paul.burt@REDACTED Fri Jul 22 09:56:03 2011 From: paul.burt@REDACTED (Paul Burt) Date: Fri, 22 Jul 2011 08:56:03 +0100 Subject: [erlang-questions] Problems with re:replace/4 In-Reply-To: <83E7466F-1823-41A5-81E4-1F12B71E0D80@gmail.com> References: <83E7466F-1823-41A5-81E4-1F12B71E0D80@gmail.com> Message-ID: Sadly not, no. I tried that. The replacement string simply returns the literal string "\k" or "\\k" no matter how many backslashes I put in! Cheers, Paul On 22 July 2011 08:51, Antoine Koener wrote: > Whenever you deal with re module and the \ character, try \\ instead. > > There's 2 levels of filtering. > > Maybe this will help :)) > > > > On 22 juil. 2011, at 09:39, Paul Burt wrote: > >> Hi, >> >> I'm a newcomer to Erlang and am interested in writing some helper >> functions to parse and manipulate responses from the Twitter API for >> use in a web application. >> >> What I would like to do is replace "@mention" and "#hashtag" with HTML >> markup as follows: >> >> @mention --> @mention >> #hashtag --> #hashtag >> >> In other words transform Twitter @mentions and #hashtags into clickable links. >> >> I have the basic structure of a function (tweetify/1) to do this >> below. The ???? represent where I'm running into trouble: >> >> tweetify(Input) -> >> ? ?Replacements = [ >> ? ? ? ?{"@(?\\w+)", "&"}, >> ? ? ? ?{"#(?\\w+)", "> href=\"http://search.twitter.com/search?tag=????\">&"} >> ? ?], >> ? ?lists:foldl(fun({RE, Replacement}, Tweet) -> >> ? ? ? ? ? ? ? ? ? ? ?re:replace(Tweet, RE, Replacement, [global, >> {return, binary}]) >> ? ? ? ? ? ? ? ? ?end, Input, Replacements). >> >> Whilst I understand that the token "&" will insert _everything_ >> matched by the regular expression (i.e. @mention and #hashtag >> respectively), how do I use the named tags in the replacement string? >> In other words, how do I get hold of \k and \k in >> the PCRE idiom and use them in the replacement string? >> >> Thanks in advance for any assistance/pointers. >> >> -Paul >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From antoine.koener@REDACTED Fri Jul 22 10:05:09 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Fri, 22 Jul 2011 10:05:09 +0200 Subject: [erlang-questions] Problems with re:replace/4 In-Reply-To: References: <83E7466F-1823-41A5-81E4-1F12B71E0D80@gmail.com> Message-ID: <005779EB-AE7B-4960-83F3-58C00CF70ED8@gmail.com> Why using 'named matches ' instead of \\1 ? since you use ( ) in the search motif ? On 22 juil. 2011, at 09:56, Paul Burt wrote: > Sadly not, no. I tried that. > > The replacement string simply returns the literal string "\k" > or "\\k" no matter how many backslashes I put in! > > Cheers, > Paul > > > On 22 July 2011 08:51, Antoine Koener wrote: >> Whenever you deal with re module and the \ character, try \\ instead. >> >> There's 2 levels of filtering. >> >> Maybe this will help :)) >> >> >> >> On 22 juil. 2011, at 09:39, Paul Burt wrote: >> >>> Hi, >>> >>> I'm a newcomer to Erlang and am interested in writing some helper >>> functions to parse and manipulate responses from the Twitter API for >>> use in a web application. >>> >>> What I would like to do is replace "@mention" and "#hashtag" with HTML >>> markup as follows: >>> >>> @mention --> @mention >>> #hashtag --> #hashtag >>> >>> In other words transform Twitter @mentions and #hashtags into clickable links. >>> >>> I have the basic structure of a function (tweetify/1) to do this >>> below. The ???? represent where I'm running into trouble: >>> >>> tweetify(Input) -> >>> Replacements = [ >>> {"@(?\\w+)", "&"}, >>> {"#(?\\w+)", ">> href=\"http://search.twitter.com/search?tag=????\">&"} >>> ], >>> lists:foldl(fun({RE, Replacement}, Tweet) -> >>> re:replace(Tweet, RE, Replacement, [global, >>> {return, binary}]) >>> end, Input, Replacements). >>> >>> Whilst I understand that the token "&" will insert _everything_ >>> matched by the regular expression (i.e. @mention and #hashtag >>> respectively), how do I use the named tags in the replacement string? >>> In other words, how do I get hold of \k and \k in >>> the PCRE idiom and use them in the replacement string? >>> >>> Thanks in advance for any assistance/pointers. >>> >>> -Paul >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paul.burt@REDACTED Fri Jul 22 10:11:31 2011 From: paul.burt@REDACTED (Paul Burt) Date: Fri, 22 Jul 2011 09:11:31 +0100 Subject: [erlang-questions] Problems with re:replace/4 In-Reply-To: <005779EB-AE7B-4960-83F3-58C00CF70ED8@gmail.com> References: <83E7466F-1823-41A5-81E4-1F12B71E0D80@gmail.com> <005779EB-AE7B-4960-83F3-58C00CF70ED8@gmail.com> Message-ID: Antoine, thanks very much, using \\N works as expected. Cheers, Paul On 22 July 2011 09:05, Antoine Koener wrote: > Why using 'named matches ' instead of \\1 ? > since you use ( ) in the search motif ? > > > > On 22 juil. 2011, at 09:56, Paul Burt wrote: > >> Sadly not, no. I tried that. >> >> The replacement string simply returns the literal string "\k" >> or "\\k" no matter how many backslashes I put in! >> >> Cheers, >> Paul >> >> >> On 22 July 2011 08:51, Antoine Koener wrote: >>> Whenever you deal with re module and the \ character, try \\ instead. >>> >>> There's 2 levels of filtering. >>> >>> Maybe this will help :)) >>> >>> >>> >>> On 22 juil. 2011, at 09:39, Paul Burt wrote: >>> >>>> Hi, >>>> >>>> I'm a newcomer to Erlang and am interested in writing some helper >>>> functions to parse and manipulate responses from the Twitter API for >>>> use in a web application. >>>> >>>> What I would like to do is replace "@mention" and "#hashtag" with HTML >>>> markup as follows: >>>> >>>> @mention --> @mention >>>> #hashtag --> #hashtag >>>> >>>> In other words transform Twitter @mentions and #hashtags into clickable links. >>>> >>>> I have the basic structure of a function (tweetify/1) to do this >>>> below. The ???? represent where I'm running into trouble: >>>> >>>> tweetify(Input) -> >>>> ? ?Replacements = [ >>>> ? ? ? ?{"@(?\\w+)", "&"}, >>>> ? ? ? ?{"#(?\\w+)", ">>> href=\"http://search.twitter.com/search?tag=????\">&"} >>>> ? ?], >>>> ? ?lists:foldl(fun({RE, Replacement}, Tweet) -> >>>> ? ? ? ? ? ? ? ? ? ? ?re:replace(Tweet, RE, Replacement, [global, >>>> {return, binary}]) >>>> ? ? ? ? ? ? ? ? ?end, Input, Replacements). >>>> >>>> Whilst I understand that the token "&" will insert _everything_ >>>> matched by the regular expression (i.e. @mention and #hashtag >>>> respectively), how do I use the named tags in the replacement string? >>>> In other words, how do I get hold of \k and \k in >>>> the PCRE idiom and use them in the replacement string? >>>> >>>> Thanks in advance for any assistance/pointers. >>>> >>>> -Paul >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From sverker@REDACTED Fri Jul 22 11:17:38 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 22 Jul 2011 11:17:38 +0200 Subject: [erlang-questions] How big can be number of ets tables? In-Reply-To: References: Message-ID: <4E294032.8010101@erix.ericsson.se> Max Lapshin wrote: > Erlang VM has option to limit number of ets tables. > > What bad will happen, if I set it to 65536? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Static meta-tables are allocated at emulator start. Currently (R14B03) no more than (ERL_MAX_ETS_TABLES * 3) words are allocated. A word is either 4 or 8 bytes depending on type of OS process. /Sverker, Erlang/OTP From watson.timothy@REDACTED Fri Jul 22 11:42:17 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 10:42:17 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: > It has been tested for several times and considered a VM OOM killer, > when you have to deal more than > with 1 MB of state. I believe this is why riak_err was created - to limit the amount of memory being consumed. > > This is why it is one of the first things, done in erlyvideo: turn off > error_logger > Or don't log massive terms - we've use error_logger in production without any issues, but large terms went to a disk_log backed thing. I do seem to remember that we found making the client (i.e., the process in which the call site appears) do the work and block seemed to offer better throughput under load. Anyway these issues about error_logger are an interesting point - perhaps Lager will be a better option. I wrote fastlog mainly for my open source projects, so it should be easy to swap out and experiment. From wsongcn@REDACTED Fri Jul 22 11:45:52 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Fri, 22 Jul 2011 17:45:52 +0800 Subject: [erlang-questions] NIF vs Erlang Binary Message-ID: Hi, I'm trying to write an Erlang WebSocket client and server against the latest spec. It requires message masking, which simply xor the whole message using a 4 bytes key. Here is the reference. I did some unit test on my code and felt that it's slow (it can process about 24M byte/s) on a virtual machine. HiPE can double the performance but still not quite enough. So I wrote an NIF to handle this. The speed is about 10~15x faster. Not only that, I feel that the C code is easier to write. Maybe it's due to my inexperienced with Erlang yet. Here is the Erlang and NIF code, FYI. Thanks Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Jul 22 11:47:22 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 10:47:22 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: On 22 July 2011 10:42, Tim Watson wrote: >> It has been tested for several times and considered a VM OOM killer, >> when you have to deal more than >> with 1 MB of state. > > I believe this is why riak_err was created - to limit the amount of > memory being consumed. > >> >> This is why it is one of the first things, done in erlyvideo: turn off >> error_logger >> So looking at erlyvideo, I notice that you're using log4erl - I take it that you've tested this and are satisfied with its performance? How did you benchmark it? From sam@REDACTED Fri Jul 22 12:15:19 2011 From: sam@REDACTED (Sam Elliott) Date: Fri, 22 Jul 2011 11:15:19 +0100 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: Add a trailing slash to your root delcaration. It's currently looking for '/var/www/files/test.html' - with the trailing slash it will look for '/var/www/test.html' (Yes, this seems like odd behaviour, but that's what they say to do on the page about it) Sam -- Sam Elliott sam@REDACTED -- On Fri, Jul 22, 2011 at 2:17 AM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > So I'm messing around with using nginx's xsendfile. I have the proxy setup > and its working fine. And I have the following in the config to serve the > static files: > > location /files { > root /var/www; > internal; > } > > The proxying works fine and when I make a request to the nginx server it > properly ends up going to the Webmachine server, which dispatch rule sends > it to my resource for static files. Here I return: > > NewReqData = wrq:set_resp_header("X-Accel-Redirect", "/files/test.html", > ReqData), > {"", NewReqData, Ctx}. > > But this does not work to end up having nginx serve up test.html from > /var/www. Should it? Am I doing it completely wrong? Any suggestions? > > Thanks, > Tristan > > On Wed, Jul 20, 2011 at 6:51 PM, Kenny Stone wrote: > >> One way to look at it is that Erlang has (more powerful) versions of >> redis/memcached built in with ETS and Mnesia. These are pretty nice >> features for web development, and I'm always a fan of cutting down the >> external dependencies for deployment and dev. Just rebar your repo and you >> get this powerful, self-contained web server executable (one of the selling >> points of couchdb, actually). >> >> Kenny >> >> >> On Wed, Jul 20, 2011 at 6:38 PM, Tristan Sloughter < >> tristan.sloughter@REDACTED> wrote: >> >>> Kenny, yeah, thats what I was thinking of doing as a cache method if I >>> couldn't use something easily in front of Webmachine like Varnish (didn't >>> actually think of Varnish until Jesper brought it up. Maybe a bit of both... >>> Since while I don't want to use any templating on the backend, I want to end >>> up with a general web framework from all this. >>> >>> Tristan >>> >>> >>> On Wed, Jul 20, 2011 at 3:47 PM, Kenny Stone wrote: >>> >>>> I wonder if an ets solution wouldn't be as good, as these solutions are >>>> just going to find some way of holding the data in memory anyways. WIth >>>> ets, you can do things like hold compiled erlydtl/mustache templates inside >>>> of it... >>>> >>>> Kenny >>>> >>>> >>>> On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: >>>> >>>>> If you're already using Nginx and just want control over URLs, you >>>>> have access to the standard rewrite module: >>>>> >>>>> http://wiki.nginx.org/HttpRewriteModule >>>>> >>>>> This is not 30x redirection btw, unless of course you want that. >>>>> >>>>> On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter >>>>> wrote: >>>>> > Sam, Jesper both these sound great, thanks! I'm also going to look >>>>> into what >>>>> > Jack was saying about how Rails handles some stuff. >>>>> > But I'm leaning towards Jesper's idea with Varnish being the best... >>>>> I'm one >>>>> > of those people who scoff at most benchmarks so not sure I'll bother >>>>> to do >>>>> > one for this, but maybe, if someone here can A) suggest the best >>>>> setup for >>>>> > it B) if it makes sense at all or would just be another worthless >>>>> benchmark >>>>> > that really gives no information about reality. >>>>> > Thanks! >>>>> > Tristan >>>>> > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen >>>>> > wrote: >>>>> >> >>>>> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >>>>> >> wrote: >>>>> >> >>>>> >> > Can anyone think of a way I can keep the nice URLs and serve the >>>>> static >>>>> >> > html >>>>> >> > files through nginx or another webserver. >>>>> >> >>>>> >> Put a Varnish accelerator in front of your system >>>>> >> (https://www.varnish-cache.org/). That way, it doesn't matter if >>>>> your >>>>> >> backend is slow at serving files as the accelerator will just cache >>>>> >> static stuff for you. In addition, you avoid the trouble of going >>>>> >> through another system as a proxy for static content. Also, the >>>>> >> solution is quite modular. On the development system, you don't need >>>>> >> more than a single system running Erlang. >>>>> >> >>>>> >> In my opinion, there is little reason not to plug into the whole >>>>> >> industry there is where the main point is to make serving HTTP go >>>>> >> faster. Trying to beat that with Erlang is probably possible, but I >>>>> >> don't think it is beneficial. Varnish is really really hard to beat. >>>>> >> It is built specifically for being insanely fast and it serves its >>>>> >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >>>>> >> blob of nasty C code. I'd rather stand on the shoulders here than >>>>> >> trying to mess with it myself. >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> J. >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > erlang-questions mailing list >>>>> > erlang-questions@REDACTED >>>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>>> > >>>>> > >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jul 22 13:03:57 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 22 Jul 2011 13:03:57 +0200 Subject: [erlang-questions] NIF vs Erlang Binary In-Reply-To: References: Message-ID: On Fri, Jul 22, 2011 at 11:45, Andy W. Song wrote: > I did some unit test on my code and felt that it's slow (it can process > about ?24M byte/s) on a virtual machine. HiPE can double the performance but > still not quite enough. So I wrote an NIF to handle this. The speed is about > 10~15x faster. Not only that, I feel that the C code is easier to write. Blindly unrolling the Key a bit gives a factor of 3 speedup: mask(Key, Data, Accu) -> K = binary:copy(<>, 512 div 32), <> = K, mask(Key, LongKey, Data, Accu). mask(Key, LongKey, Data,Accu) -> case Data of <> -> C = binary:encode_unsigned(A bxor LongKey), mask(Key, LongKey, Rest, <>); <> -> C = binary:encode_unsigned(A bxor Key), mask(Key,LongKey,Rest,<>); <> -> <> = binary:encode_unsigned(Key), C = binary:encode_unsigned(A bxor B), <>; <> -> <> = binary:encode_unsigned(Key), C = binary:encode_unsigned(A bxor B), <>; <> -> <> = binary:encode_unsigned(Key), C = binary:encode_unsigned(A bxor B), <>; <<>> -> Accu end. Why the call to binary:encode_unsigned? Lets alter that pattern: case Data of <> -> C = A bxor LongKey, mask(Key, LongKey, Rest, <>); Now it is 5 times faster, same result. The NIF-advantage is now a factor of 2-3. That is in the ballpark I would expect it to be. You are doing many more reallocations with the above solution. Then the C NIF version. What happens if we tune it some more? Lets do runs of 8192 bits at a time... 9 times faster compared to the original here! I expect our speed will converge to that of C if we turn it up even more and get the amount of allocation/realloc/concatenation down. -- J. From dmercer@REDACTED Fri Jul 22 15:10:24 2011 From: dmercer@REDACTED (David Mercer) Date: Fri, 22 Jul 2011 08:10:24 -0500 Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: <218172986.121321311212519505.JavaMail.root@zimbra> References: <218172986.121321311212519505.JavaMail.root@zimbra> Message-ID: <001501cc4870$b901ed30$2b05c790$@com> I was curious about that, too. Hoping you'll get a response... > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] On Behalf Of Robert Virding > Sent: Wednesday, July 20, 2011 8:42 PM > To: Parnell Springmeyer > Cc: erlang-questions > Subject: Re: [erlang-questions] Process pool map/3 implementation > > One quick question: what was wrong with the straightforward solution of > just spawning one process for each element in the list? Did this break > or do you actually need more control? > > Robert > > ----- "Parnell Springmeyer" wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > For a work project I have a large list (thousands of items) to > > process > > and at first built a "pmap" implementation as per Joe's book until I > > found the plists module (which is awesome btw). > > > > There is one glaring issue with the list -> subdivide -> spawn x > > processes for n sublist items strategy; if an item in the sublist > > takes > > longer than all the other items it blocks the entire resource > > allotment > > until it is done. > > > > In most cases, the plists/pmap implementation works just fine because > > the items in the list probably don't take more than a few > > milliseconds > > to map the fun over. However, it does become an issue when that is > > not > > the case. > > > > So, I figured the next best strategy would be to implement a process > > pool since it would allow for slow running processes to continue > > their > > work while finished processes can die and new processes spawned into > > the > > pool ready for work - so none of the resources are sitting idle. > > > > Right now, my module isn't nearly as feature-complete as the plists > > module is - this is only a drop in replacement for map. Please submit > > your criticisms and comments to me at this address. > > > > You may find the code on BitBucket: > > https://bitbucket.org/ixmatus/ppool > > > > - -- > > Parnell "ixmatus" Springmeyer (http://ixmat.us) > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > > Comment: GPGTools - http://gpgtools.org > > > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > > =GP7K > > -----END PGP SIGNATURE----- > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Jul 22 15:47:29 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 22 Jul 2011 17:47:29 +0400 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: On Friday, July 22, 2011, Tim Watson wrote: > On 22 July 2011 10:42, Tim Watson wrote: >>> It has been tested for several times and considered a VM OOM killer, >>> when you have to deal more than >>> with 1 MB of state. >> >> I believe this is why riak_err was created - to limit the amount of >> memory being consumed. >> >>> >>> This is why it is one of the first things, done in erlyvideo: turn off >>> error_logger >>> > > So looking at erlyvideo, I notice that you're using log4erl - I take > it that you've tested this and are satisfied with its performance? How > did you benchmark it? > It just works because erlyvideo serves 300-4000 simultaneous clients. They provide not too much logging events. When error_logger brings down erlang vm, it happens because of wrong matching with large state, or dumping message queue or last message. That is why gen_server also need fix From tristan.sloughter@REDACTED Fri Jul 22 15:50:57 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Fri, 22 Jul 2011 08:50:57 -0500 Subject: [erlang-questions] Static files served through Webmachine In-Reply-To: References: Message-ID: So like: root /var/www/ Didn't seem to work. I need to find where OSX stores the damn nginx logs... That will probably explain whats going on. On Fri, Jul 22, 2011 at 5:15 AM, Sam Elliott wrote: > Add a trailing slash to your root delcaration. It's currently looking for > '/var/www/files/test.html' - with the trailing slash it will look for > '/var/www/test.html' (Yes, this seems like odd behaviour, but that's what > they say to do on the page about it) > > Sam > -- > Sam Elliott > sam@REDACTED > -- > > > > On Fri, Jul 22, 2011 at 2:17 AM, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> So I'm messing around with using nginx's xsendfile. I have the proxy setup >> and its working fine. And I have the following in the config to serve the >> static files: >> >> location /files { >> root /var/www; >> internal; >> } >> >> The proxying works fine and when I make a request to the nginx server it >> properly ends up going to the Webmachine server, which dispatch rule sends >> it to my resource for static files. Here I return: >> >> NewReqData = wrq:set_resp_header("X-Accel-Redirect", "/files/test.html", >> ReqData), >> {"", NewReqData, Ctx}. >> >> But this does not work to end up having nginx serve up test.html from >> /var/www. Should it? Am I doing it completely wrong? Any suggestions? >> >> Thanks, >> Tristan >> >> On Wed, Jul 20, 2011 at 6:51 PM, Kenny Stone wrote: >> >>> One way to look at it is that Erlang has (more powerful) versions of >>> redis/memcached built in with ETS and Mnesia. These are pretty nice >>> features for web development, and I'm always a fan of cutting down the >>> external dependencies for deployment and dev. Just rebar your repo and you >>> get this powerful, self-contained web server executable (one of the selling >>> points of couchdb, actually). >>> >>> Kenny >>> >>> >>> On Wed, Jul 20, 2011 at 6:38 PM, Tristan Sloughter < >>> tristan.sloughter@REDACTED> wrote: >>> >>>> Kenny, yeah, thats what I was thinking of doing as a cache method if I >>>> couldn't use something easily in front of Webmachine like Varnish (didn't >>>> actually think of Varnish until Jesper brought it up. Maybe a bit of both... >>>> Since while I don't want to use any templating on the backend, I want to end >>>> up with a general web framework from all this. >>>> >>>> Tristan >>>> >>>> >>>> On Wed, Jul 20, 2011 at 3:47 PM, Kenny Stone wrote: >>>> >>>>> I wonder if an ets solution wouldn't be as good, as these solutions are >>>>> just going to find some way of holding the data in memory anyways. WIth >>>>> ets, you can do things like hold compiled erlydtl/mustache templates inside >>>>> of it... >>>>> >>>>> Kenny >>>>> >>>>> >>>>> On Tue, Jul 19, 2011 at 11:47 AM, Garrett Smith wrote: >>>>> >>>>>> If you're already using Nginx and just want control over URLs, you >>>>>> have access to the standard rewrite module: >>>>>> >>>>>> http://wiki.nginx.org/HttpRewriteModule >>>>>> >>>>>> This is not 30x redirection btw, unless of course you want that. >>>>>> >>>>>> On Tue, Jul 19, 2011 at 9:24 AM, Tristan Sloughter >>>>>> wrote: >>>>>> > Sam, Jesper both these sound great, thanks! I'm also going to look >>>>>> into what >>>>>> > Jack was saying about how Rails handles some stuff. >>>>>> > But I'm leaning towards Jesper's idea with Varnish being the best... >>>>>> I'm one >>>>>> > of those people who scoff at most benchmarks so not sure I'll bother >>>>>> to do >>>>>> > one for this, but maybe, if someone here can A) suggest the best >>>>>> setup for >>>>>> > it B) if it makes sense at all or would just be another worthless >>>>>> benchmark >>>>>> > that really gives no information about reality. >>>>>> > Thanks! >>>>>> > Tristan >>>>>> > On Tue, Jul 19, 2011 at 7:00 AM, Jesper Louis Andersen >>>>>> > wrote: >>>>>> >> >>>>>> >> On Tue, Jul 19, 2011 at 03:27, Tristan Sloughter >>>>>> >> wrote: >>>>>> >> >>>>>> >> > Can anyone think of a way I can keep the nice URLs and serve the >>>>>> static >>>>>> >> > html >>>>>> >> > files through nginx or another webserver. >>>>>> >> >>>>>> >> Put a Varnish accelerator in front of your system >>>>>> >> (https://www.varnish-cache.org/). That way, it doesn't matter if >>>>>> your >>>>>> >> backend is slow at serving files as the accelerator will just cache >>>>>> >> static stuff for you. In addition, you avoid the trouble of going >>>>>> >> through another system as a proxy for static content. Also, the >>>>>> >> solution is quite modular. On the development system, you don't >>>>>> need >>>>>> >> more than a single system running Erlang. >>>>>> >> >>>>>> >> In my opinion, there is little reason not to plug into the whole >>>>>> >> industry there is where the main point is to make serving HTTP go >>>>>> >> faster. Trying to beat that with Erlang is probably possible, but I >>>>>> >> don't think it is beneficial. Varnish is really really hard to >>>>>> beat. >>>>>> >> It is built specifically for being insanely fast and it serves its >>>>>> >> data from a shared mmap()'ing, scales to multiple CPUs and is a big >>>>>> >> blob of nasty C code. I'd rather stand on the shoulders here than >>>>>> >> trying to mess with it myself. >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> J. >>>>>> > >>>>>> > >>>>>> > _______________________________________________ >>>>>> > erlang-questions mailing list >>>>>> > erlang-questions@REDACTED >>>>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>>>> > >>>>>> > >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Fri Jul 22 15:58:30 2011 From: mihai@REDACTED (Mihai Balea) Date: Fri, 22 Jul 2011 09:58:30 -0400 Subject: [erlang-questions] NIF vs Erlang Binary In-Reply-To: References: Message-ID: <061C60FA-D72B-40A1-AE6E-4BC49B944936@hates.ms> On Jul 22, 2011, at 7:03 AM, Jesper Louis Andersen wrote: > On Fri, Jul 22, 2011 at 11:45, Andy W. Song wrote: > >> I did some unit test on my code and felt that it's slow (it can process >> about 24M byte/s) on a virtual machine. HiPE can double the performance but >> still not quite enough. So I wrote an NIF to handle this. The speed is about >> 10~15x faster. Not only that, I feel that the C code is easier to write. > > Blindly unrolling the Key a bit gives a factor of 3 speedup: > > Now it is 5 times faster, same result. The NIF-advantage is now a > factor of 2-3. That is in the ballpark I would expect it to be. You > are doing many more reallocations with the above solution. Then the C > NIF version. What happens if we tune it some more? Lets do runs of > 8192 bits at a time... > > 9 times faster compared to the original here! I expect our speed will > converge to that of C if we turn it up even more and get the amount of > allocation/realloc/concatenation down. Just for fun, I did a test with a version of mask based on binary comprehensions. Here's the code: mask1(Key, Data) -> S = size(Data) div 4 * 4, <> = Data, D2 = << <<(X bxor Key):32>> || <> <= D1 >>, T2 = handle_tail(T1, <>), <>. handle_tail(<>, <>) -> <<(A bxor K):24>>; handle_tail(<>, <>) -> <<(A bxor K):16>>; handle_tail(<>, <>) -> <<(A bxor K):8>>; handle_tail(<<>>, _) -> <<>>. The speed gain is disappointingly small, it shaves roughly 40% of Andy's original times. I suspect that's because the comprehension is just syntactic sugar on top of recursive loops. Anyway, just thought I'd share the results :) Mihai From watson.timothy@REDACTED Fri Jul 22 16:21:09 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 15:21:09 +0100 Subject: [erlang-questions] [ANN] Lager - a new logging framework for Erlang/OTP In-Reply-To: References: <20110720211917.GA16695@hijacked.us> <20110722014100.GB9340@hijacked.us> Message-ID: > > It just works because erlyvideo serves 300-4000 simultaneous clients. > They provide not too much logging events. > Cool - I'm quite interested in comparing the three (mentioned) so maybe instead of taking my logging framework beyond its current (very early) stage, I will write a little library to provide a logging API and swap out at compile and/or runtime the actual logging framework being used. That might be more useful in general than yet another logging library. > > When error_logger brings down erlang vm, it happens because of wrong > matching with large state, or dumping message queue or last message. > > That is why gen_server also need fix > Ok that's good to know - thanks for pointing this out. From sean@REDACTED Fri Jul 22 17:20:53 2011 From: sean@REDACTED (Sean Moss-Pultz) Date: Fri, 22 Jul 2011 23:20:53 +0800 Subject: [erlang-questions] compiling problems under OS X Lion Message-ID: Hi List I'm trying to build?R14B03 under OS X Lion. (Xcode v4.1). I'm getting the following error: -------------------------------------------------- === Entering application hipe (cd ../main && make hipe.hrl) sed -e "s;%VSN%;3.8;" ../../hipe/main/hipe.hrl.src > ../../hipe/main/hipe.hrl erlc -W ?+debug_info +inline -o../ebin hipe_rtl.erl (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<47 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<12 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<50 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<6 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<12 bytes>>]},[{erlang,apply,2}]}\n" -------------------------------------------------- People using homebrew seem to have the same issue: https://github.com/mxcl/homebrew/issues/2713 They say to use --use-gcc when running ./configure. I've tried this but it didn't change the error. Can somebody help me with this? Thanks! ? -Sean From dizzyd@REDACTED Fri Jul 22 17:37:58 2011 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 22 Jul 2011 09:37:58 -0600 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: Message-ID: If you want to compile Erlang w/ clang-llvm on 10.6 (or above) do this: CFLAGS=-O0 ./configure ... D. On Fri, Jul 22, 2011 at 9:20 AM, Sean Moss-Pultz wrote: > Hi List > > I'm trying to build?R14B03 under OS X Lion. (Xcode v4.1). I'm getting > the following error: > > -------------------------------------------------- > > === Entering application hipe > (cd ../main && make hipe.hrl) > sed -e "s;%VSN%;3.8;" ../../hipe/main/hipe.hrl.src > ../../hipe/main/hipe.hrl > erlc -W ?+debug_info +inline -o../ebin hipe_rtl.erl > (no error logger present) error: "Error in process <0.1.0> with exit > value: {{badfun,[<<5 bytes>>,<<47 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 > bytes>>,<<5 bytes>>,<<12 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 > bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 > bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<50 bytes>>,<<5 > bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<6 > bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<12 > bytes>>]},[{erlang,apply,2}]}\n" > > -------------------------------------------------- > > People using homebrew seem to have the same issue: > > ?https://github.com/mxcl/homebrew/issues/2713 > > They say to use --use-gcc when running ./configure. I've tried this > but it didn't change the error. Can somebody help me with this? > > Thanks! > > > ? -Sean > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ngreco@REDACTED Fri Jul 22 18:10:28 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Fri, 22 Jul 2011 13:10:28 -0300 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <8327DB12-48D3-486F-9736-535CC289F5B0@cs.otago.ac.nz> <00E93C90-7026-4014-BCFB-234CDDF85944@cs.otago.ac.nz> Message-ID: On Fri, Jul 22, 2011 at 4:55 AM, Joe Armstrong wrote: > On Fri, Jul 22, 2011 at 6:28 AM, Richard O'Keefe wrote: > > I have three javascript books at home - I looked up assignment in all three. > But none of them said anything sensible ... > > What does > ? ? x = y; > mean in javascript? > If I say > ? ? x = y; > In javascript and then change y, is x changed? Or if I change x is > y changed. > Javascript books are terrible, you really need to study the ECMA-262 document to get it. Closures in JS doesn't bind the values of the outer lexical context variables when they are created. Instead, they keep a reference to the outer scope frame. So when you execute the closure, there is a level of indirection to the value not present in Erlang. Inside the closure you are really doing something like outer_scope['x'] = y; Saludos, Nahuel Greco From robert.virding@REDACTED Fri Jul 22 20:15:54 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 22 Jul 2011 18:15:54 +0000 (GMT) Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <1783425682.129231311358461747.JavaMail.root@zimbra> Message-ID: <20229112.129251311358554198.JavaMail.root@zimbra> Just a quick comment on Erlang fun syntax, without getting into a discussion on Tony's suggestion. This is how I remember it. The fun syntax was chosen to look like the syntax for "normal" functions. This because they basically have the same properties and features: multiple clauses, patterns, guards, etc. Seeing they basically behave in the same way they should look the same way. Now a choice was made to start them with the reserved word 'fun' and not to repeat it for every clause. I can't remember why we chose not to repeat the "function name" (fun) for each clause but my guess is that we considered it redundant so we chose not to include it. Another choice we made was that all variables occurring in the head of a fun clause is a new variable which inside the fun clause shadows variables with the same name outside the fun. This means, of course, that any comparison with variables outside has to be done explicitly in the guard. This is, of course, un-Erlangy but I guess we thought it fitted better into the "standard" way of the fun. We couldn't, of course, end a fun definition with a '.' as this would really screw up the syntax handling so we ended a fun in the same was as we end everything, except a catch (this was a mistake, it would have been better to have catch ... end), with an 'end'. I know the syntax is a bit wordy but at least both the syntax and semantics are consistent. (And at my age I value consistency) The handling of variables was then "imported" into list comprehensions, so that there now are two things which treat variables differently. I very much like the idea of a proper let (or a where) and I think it would make variable handling clearer. Variables are handled in a consistent way but it is not always easy to work out what should happen. Most people never ave problems with this as most people write pretty "simple" code (from the compiler point of view). Adding local functions, which I think would be a Good Thing, in the same way as variables are handled today would be very confusing so some form of 'let' construct would be necessary, at least for them. Then we might as well have for variables as well. Sorry for digressing a little here. Robert ----- "Joe Armstrong" wrote: > On Fri, Jul 22, 2011 at 6:28 AM, Richard O'Keefe > wrote: > > > > On 22/07/2011, at 3:49 PM, Tony Arcieri wrote: > > > >> On Thu, Jul 21, 2011 at 7:43 PM, Richard O'Keefe > wrote: > >> Wrong. ?In funs, "end" is paired with "fun" and "fun" ONLY. > >> > >> Okay, conceded (and Jachym made the same point) > >> > >> I'll revert to my other point, which is that the '->' token in > Erlang is only found in forms outside of funs, which are expressions. > No other expression in Erlang uses the '->' token. > > > > I am totally confused here. > > Erlang constructs "->" if and only if they have patterns-and-guards > > (well, patterns-and-or-guards, to be picky) to separate from > bodies. > > > > That's it. ?No exceptions. > > > > I have just revised the Expressions chapter of the reference > manual, > > and I cannot find one single example of a kind of expression that > > _should_ use "->" but doesn't. > > > > So what kind of expression do _you_ think ought to use "->" but > doesn't? > > > >> I'd also ask what you feel about the people who responded to this > thread > >> who find the Erlang fun syntax awkward. I certainly find it > awkward. > > > > Oh dear, this is going to sound insulting, but it's not meant that > way. > > > > I am more impressed by people who demonstrate that they understand > how > > the current syntax actually *works* before they complain about it. > > > > I myself have said that I find Erlang syntax "clunky but tasteful". > > It uses more tokens than Haskell does. > > > > But then, Haskell syntax is surprisingly tricky, and I'm not sure > > how many Haskell parsers actually get it 100% right. > > > > For reference, it took a huge amount of fighting in the ISO Prolog > > committee to agree on an unambiguous syntax, and in the revision > mailing > > list there are continuing, um, strong disagreements. ?I think the > > restrictions compared with classic Prolog were and are too severe. > ?But > > a correspondent who has done a heck of a lot more work on standards > issues > > in recent years than I have points out with a great deal of justice > that > > the various Prolog implementations out there STILL don't agree on > syntax, > > not as much as their authors fondly imagine. ?He'd like to see them > get > > what's in the standard right before adding anything else. > > > > Do not despise a simple syntax that implementors can actually get > RIGHT. > > > > It is *HARD* to come up with a clean coherent syntax for the whole > of a > > programming language. ?And it is triply hard to patch your way > there, > > and outright impossible to do it by adding inconsistencies. > > > > Yes, "fun ... end" is not as pretty as it could be, but for > something > > that was added years after Erlang was shipping, it's nearly as > close > > to the rest of the language as anyone could wish for. ?(The thing I > > don't like is the scope rules for variables shared between "fun" > heads > > and their contexts, which has nothing to do with keywords or > > punctuation.) > > > > If you don't like funs in Erlang, try Objective C or JavaScript for > a while. > > Now you've set me off ... > > If you think Erlang funs are problematic you should take a look > at javascript. Javascript closures are truly horrible > > NOTE in what follows "$$" means the javascript shell prompt > > $$ x = 10; > 10 > > Now let's make a function y that adds x to its argument > Pretty simply, eh, > > In Erlang I'd write fun(Y) -> Y + X end > > In Javascript I write: > > $$ f = function(y){return(x+y);}; > function (y){return(x+y);} > > Test it: > > $$ f(5); > 15 > > Hooray - great - but wait a moment > > Now change x > > $$ x=20; > 20 > > Now if the force was with javascript I'd rather hope that > I had not accidentally broken f. What happens? > > Now what was f(5), let's take a look: > > $$ f(5); > 25 > > Oh dear - maths is broken. f(5) is not what it was earlier > > Try again: > > $$ x = 10; > 10 > > Add an extra level of indirection and throw in some semicolons > > $$ f = (function(z){return function(y){return y+z;};})(x); > > (( F = fun(Y) -> Y + X end, in Erlang :-)) > > > function (y){return y+z} > $$ f(5); > 15 > > So far so good. > > Now lets change x > > $$ x=20; > 20 > > Have we broken f? > > $$ f(5); > 15 > > No f still works. > > No Lets' go back to Erlang and repeat the argument as it were. > Let's pretent Erlang is javascript > > > X = 10; > > F = fun(Y) -> X + Y end; > > Now ask the javascript question "what happens if I now change X" > Erlang answer - you can't > > Now if you think horrible incantations like > > f = (function(z){return function(y){return y+z;};})(x); > > in Javascript solved all your problems with closures think again. What > you > do inside the {...} bits of a function depends crucially on the > semantics > of assignment in javascript. > > I have three javascript books at home - I looked up assignment in all > three. > But none of them said anything sensible ... > > What does > > x = y; > > mean in javascript? > > If I say > > x = y; > > In javascript and then change y, is x changed? Or if I change x is > y changed. > > Is x a deep or shallow copy of y - well the answer is of course "it > depends" > Now in Erlang X = Y behaves as if you'd made a deep copy of Y. > Of course you don't actually make a deep copy of Y - you just think > you have, > and the question "what happens to X if we subsequently change Y?" is > easy to answer "You can't change Y" > > (actually - non of the three Erlang books mention this, if you come > from javascript > you have to worry about the semantics of assignment, and the deep and > shallow copying implied by assignment) > > Erlang fun syntax might have the odd wart (variable bindings can > sometimes > seem strange to the unaccustomed eye) but at least things bound in > closures > are actually bound in the closure and cannot be changed later - this > means no nasty surprises later. > > Cheers > > /Joe > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Fri Jul 22 20:23:44 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 22 Jul 2011 19:23:44 +0100 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <20229112.129251311358554198.JavaMail.root@zimbra> References: <1783425682.129231311358461747.JavaMail.root@zimbra> <20229112.129251311358554198.JavaMail.root@zimbra> Message-ID: > I very much like the idea of a proper let (or a where) and I think it would make variable handling clearer. Variables are handled in a consistent way but it is not always easy to work out what should happen. Most people never ave problems with this as most people write pretty "simple" code (from the compiler point of view). Adding local functions, which I think would be a Good Thing, in the same way as variables are handled today would be very confusing so some form of 'let' construct would be necessary, at least for them. Then we might as well have for variables as well. > I like the idea of let bindings - I think someone has suggested this before. It might have been Richard, although apologies if I've got that wrong. From tristan.sloughter@REDACTED Fri Jul 22 20:35:42 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Fri, 22 Jul 2011 13:35:42 -0500 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <1783425682.129231311358461747.JavaMail.root@zimbra> <20229112.129251311358554198.JavaMail.root@zimbra> Message-ID: I think everyone has suggested let bindings at some point in time for Erlang :) On Fri, Jul 22, 2011 at 1:23 PM, Tim Watson wrote: > > I very much like the idea of a proper let (or a where) and I think it > would make variable handling clearer. Variables are handled in a consistent > way but it is not always easy to work out what should happen. Most people > never ave problems with this as most people write pretty "simple" code (from > the compiler point of view). Adding local functions, which I think would be > a Good Thing, in the same way as variables are handled today would be very > confusing so some form of 'let' construct would be necessary, at least for > them. Then we might as well have for variables as well. > > > > I like the idea of let bindings - I think someone has suggested this > before. It might have been Richard, although apologies if I've got > that wrong. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ixmatus@REDACTED Sat Jul 23 01:02:43 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Fri, 22 Jul 2011 16:02:43 -0700 Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: <001501cc4870$b901ed30$2b05c790$@com> References: <218172986.121321311212519505.JavaMail.root@zimbra> <001501cc4870$b901ed30$2b05c790$@com> Message-ID: Because the list has about 3000 items in it, and for each item about 20-50 HTTP requests are made; I needed a way of parallelizing the operations (instead of stepping through the list one by one) but in a controlled fashion and using a round robin strategy (worker pool). On Fri, Jul 22, 2011 at 6:10 AM, David Mercer wrote: > I was curious about that, too. Hoping you'll get a response... > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > > bounces@REDACTED] On Behalf Of Robert Virding > > Sent: Wednesday, July 20, 2011 8:42 PM > > To: Parnell Springmeyer > > Cc: erlang-questions > > Subject: Re: [erlang-questions] Process pool map/3 implementation > > > > One quick question: what was wrong with the straightforward solution of > > just spawning one process for each element in the list? Did this break > > or do you actually need more control? > > > > Robert > > > > ----- "Parnell Springmeyer" wrote: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > For a work project I have a large list (thousands of items) to > > > process > > > and at first built a "pmap" implementation as per Joe's book until I > > > found the plists module (which is awesome btw). > > > > > > There is one glaring issue with the list -> subdivide -> spawn x > > > processes for n sublist items strategy; if an item in the sublist > > > takes > > > longer than all the other items it blocks the entire resource > > > allotment > > > until it is done. > > > > > > In most cases, the plists/pmap implementation works just fine because > > > the items in the list probably don't take more than a few > > > milliseconds > > > to map the fun over. However, it does become an issue when that is > > > not > > > the case. > > > > > > So, I figured the next best strategy would be to implement a process > > > pool since it would allow for slow running processes to continue > > > their > > > work while finished processes can die and new processes spawned into > > > the > > > pool ready for work - so none of the resources are sitting idle. > > > > > > Right now, my module isn't nearly as feature-complete as the plists > > > module is - this is only a drop in replacement for map. Please submit > > > your criticisms and comments to me at this address. > > > > > > You may find the code on BitBucket: > > > https://bitbucket.org/ixmatus/ppool > > > > > > - -- > > > Parnell "ixmatus" Springmeyer (http://ixmat.us) > > > -----BEGIN PGP SIGNATURE----- > > > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > > > Comment: GPGTools - http://gpgtools.org > > > > > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > > > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > > > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > > > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > > > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > > > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > > > =GP7K > > > -----END PGP SIGNATURE----- > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat Jul 23 02:10:47 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 23 Jul 2011 00:10:47 +0000 (GMT) Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: Message-ID: <45993791.129661311379846617.JavaMail.root@zimbra> But a pmap SHOULD start processes for all the elements in the list in parallel. It is after all a 'P'map. In which case all the processes will be running and processing in parallel as you want. The only reason I can see for using a worker pool is if you actually want to LIMIT the number of processes running at the same time. IMAO in Erlang there are only two reasons for using worker/process pools: - you want/need to limit the number of "things" running in parallel - you actually do want to reuse a process for another computation, there is something in the application which mandates reusing processes. Otherwise it is just extra work, process creation/termination is so fast that there is no real gain in keeping them around to reuse. Robert ----- "Parnell Springmeyer" wrote: > Because the list has about 3000 items in it, and for each item about 20-50 HTTP requests are made; I needed a way of parallelizing the operations (instead of stepping through the list one by one) but in a controlled fashion and using a round robin strategy (worker pool). > > On Fri, Jul 22, 2011 at 6:10 AM, David Mercer < dmercer@REDACTED > wrote: > I was curious about that, too. Hoping you'll get a response... > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED [mailto: erlang-questions- > > bounces@REDACTED ] On Behalf Of Robert Virding > > Sent: Wednesday, July 20, 2011 8:42 PM > > To: Parnell Springmeyer > > Cc: erlang-questions > > Subject: Re: [erlang-questions] Process pool map/3 implementation > > > > One quick question: what was wrong with the straightforward solution of > > just spawning one process for each element in the list? Did this break > > or do you actually need more control? > > > > Robert > > > > > ----- "Parnell Springmeyer" < ixmatus@REDACTED > wrote: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > For a work project I have a large list (thousands of items) to > > > process > > > and at first built a "pmap" implementation as per Joe's book until I > > > found the plists module (which is awesome btw). > > > > > > There is one glaring issue with the list -> subdivide -> spawn x > > > processes for n sublist items strategy; if an item in the sublist > > > takes > > > longer than all the other items it blocks the entire resource > > > allotment > > > until it is done. > > > > > > In most cases, the plists/pmap implementation works just fine because > > > the items in the list probably don't take more than a few > > > milliseconds > > > to map the fun over. However, it does become an issue when that is > > > not > > > the case. > > > > > > So, I figured the next best strategy would be to implement a process > > > pool since it would allow for slow running processes to continue > > > their > > > work while finished processes can die and new processes spawned into > > > the > > > pool ready for work - so none of the resources are sitting idle. > > > > > > Right now, my module isn't nearly as feature-complete as the plists > > > module is - this is only a drop in replacement for map. Please submit > > > your criticisms and comments to me at this address. > > > > > > You may find the code on BitBucket: > > > https://bitbucket.org/ixmatus/ppool > > > > > > - -- > > > Parnell "ixmatus" Springmeyer ( http://ixmat.us ) > > > -----BEGIN PGP SIGNATURE----- > > > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > > > Comment: GPGTools - http://gpgtools.org > > > > > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS > > > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV > > > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S > > > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW > > > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA > > > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= > > > =GP7K > > > -----END PGP SIGNATURE----- > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Parnell "ixmatus" Springmeyer ( http://ixmat.us ) > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Sat Jul 23 03:03:10 2011 From: sean@REDACTED (Sean Moss-Pultz) Date: Sat, 23 Jul 2011 09:03:10 +0800 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: Message-ID: > On Fri, Jul 22, 2011 at 9:20 AM, Sean Moss-Pultz wrote: >> Hi List >> >> I'm trying to build?R14B03 under OS X Lion. (Xcode v4.1). I'm getting >> the following error: {snip} On Fri, Jul 22, 2011 at 11:37 PM, Dave Smith wrote: > If you want to compile Erlang w/ clang-llvm on 10.6 (or above) do > this: CFLAGS=-O0 ./configure ... > Thanks Dave. That got me a lot further. But I'm still getting the following errors: ------------------------------------------------------------------------------------------------------------------ gcc -O0 -no-cpp-precomp -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Winline -fno-strict-aliasing -I. -I../include -Iconnect -Iencode -Idecode -Imisc -Iepmd -Iregistry -Ii386-apple-darwin11.0.0 -Ilegacy -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -m32 -o /Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/bin/i386-apple-darwin11.0.0/erl_call prog/erl_call.c prog/erl_start.c \ -L/Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/obj/i386-apple-darwin11.0.0 -lei -lpthread ld: warning: ignoring file /Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/obj/i386-apple-darwin11.0.0/libei.a, file was built for archive which is not the architecture being linked (i386) Undefined symbols for architecture i386: "_ei_gethostbyname", referenced from: _main in ccO2Es4f.o _do_connect in ccO2Es4f.o "_ei_connect_xinit", referenced from: _main in ccO2Es4f.o "___erl_errno_place", referenced from: _main in ccO2Es4f.o "_ei_thishostname", referenced from: _main in ccO2Es4f.o _exec_erlang in cc6g0Be0.o "_ei_connect", referenced from: _main in ccO2Es4f.o _do_connect in ccO2Es4f.o "_ei_encode_list_header", referenced from: _main in ccO2Es4f.o "_ei_x_new_with_version", referenced from: _main in ccO2Es4f.o "_ei_rpc", referenced from: _main in ccO2Es4f.o "_ei_x_free", referenced from: _main in ccO2Es4f.o "_ei_encode_string", referenced from: _main in ccO2Es4f.o "_ei_encode_binary", referenced from: _main in ccO2Es4f.o "_ei_encode_atom", referenced from: _main in ccO2Es4f.o "_ei_print_term", referenced from: _main in ccO2Es4f.o "_ei_x_new", referenced from: _main in ccO2Es4f.o "_ei_x_format_wo_ver", referenced from: _main in ccO2Es4f.o "_ei_gethostbyname_r", referenced from: _get_addr in cc6g0Be0.o ld: symbol(s) not found for architecture i386 collect2: ld returned 1 exit status make[4]: *** [/Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/bin/i386-apple-darwin11.0.0/erl_call] Error 1 make[3]: *** [opt] Error 2 make[2]: *** [opt] Error 2 make[1]: *** [opt] Error 2 make: *** [libs] Error 2 ------------------------------------------------------------------------------------------------------------------ Any ideas on this one? I should add, Lion was an upgrade (from Snow Leopard), not a new install. Thanks! Sean From dizzyd@REDACTED Sat Jul 23 04:06:26 2011 From: dizzyd@REDACTED (Dave Smith) Date: Fri, 22 Jul 2011 20:06:26 -0600 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: Message-ID: Try re-doing configure with --enable-darwin-64bit; note that you may need to wipe out the otp_src directory to get a clean build. D. On Fri, Jul 22, 2011 at 7:03 PM, Sean Moss-Pultz wrote: >> On Fri, Jul 22, 2011 at 9:20 AM, Sean Moss-Pultz wrote: >>> Hi List >>> >>> I'm trying to build?R14B03 under OS X Lion. (Xcode v4.1). I'm getting >>> the following error: > > {snip} > > On Fri, Jul 22, 2011 at 11:37 PM, Dave Smith wrote: >> If you want to compile Erlang w/ clang-llvm on 10.6 (or above) do >> this: CFLAGS=-O0 ./configure ... >> > > Thanks Dave. That got me a lot further. But I'm still getting the > following errors: > > ------------------------------------------------------------------------------------------------------------------ > > gcc -O0 -no-cpp-precomp -Wall -Wstrict-prototypes -Wmissing-prototypes > -Wmissing-declarations -Wnested-externs -Winline -fno-strict-aliasing > -I. -I../include -Iconnect -Iencode -Idecode -Imisc -Iepmd -Iregistry > -Ii386-apple-darwin11.0.0 ? -Ilegacy -D_THREAD_SAFE -D_REENTRANT > -DPOSIX_THREADS -m32 ?-o > /Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/bin/i386-apple-darwin11.0.0/erl_call > prog/erl_call.c prog/erl_start.c \ > ? ? ? ? ? ? ? ?-L/Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/obj/i386-apple-darwin11.0.0 > -lei -lpthread > ld: warning: ignoring file > /Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/obj/i386-apple-darwin11.0.0/libei.a, > file was built for archive which is not the architecture being linked > (i386) > Undefined symbols for architecture i386: > ?"_ei_gethostbyname", referenced from: > ? ? ?_main in ccO2Es4f.o > ? ? ?_do_connect in ccO2Es4f.o > ?"_ei_connect_xinit", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"___erl_errno_place", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_thishostname", referenced from: > ? ? ?_main in ccO2Es4f.o > ? ? ?_exec_erlang in cc6g0Be0.o > ?"_ei_connect", referenced from: > ? ? ?_main in ccO2Es4f.o > ? ? ?_do_connect in ccO2Es4f.o > ?"_ei_encode_list_header", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_x_new_with_version", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_rpc", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_x_free", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_encode_string", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_encode_binary", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_encode_atom", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_print_term", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_x_new", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_x_format_wo_ver", referenced from: > ? ? ?_main in ccO2Es4f.o > ?"_ei_gethostbyname_r", referenced from: > ? ? ?_get_addr in cc6g0Be0.o > ld: symbol(s) not found for architecture i386 > collect2: ld returned 1 exit status > make[4]: *** [/Users/mosko/Downloads/otp_src_R14B03/lib/erl_interface/bin/i386-apple-darwin11.0.0/erl_call] > Error 1 > make[3]: *** [opt] Error 2 > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [libs] Error 2 > > ------------------------------------------------------------------------------------------------------------------ > > Any ideas on this one? I should add, Lion was an upgrade (from Snow > Leopard), not a new install. > > Thanks! > > Sean > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From sean@REDACTED Sat Jul 23 05:44:08 2011 From: sean@REDACTED (Sean Moss-Pultz) Date: Sat, 23 Jul 2011 11:44:08 +0800 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: Message-ID: On Sat, Jul 23, 2011 at 10:06 AM, Dave Smith wrote: > Try re-doing configure with --enable-darwin-64bit; note that you may > need to wipe out the otp_src directory to get a clean build. Thanks a lot Dave. That solved the problem! Sean From jesper.louis.andersen@REDACTED Sat Jul 23 12:58:45 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 23 Jul 2011 12:58:45 +0200 Subject: [erlang-questions] Anyone using PropEr on their project yet? In-Reply-To: <4E2764D7.4070505@dev-extend.eu> References: <4E2764D7.4070505@dev-extend.eu> Message-ID: On Thu, Jul 21, 2011 at 01:29, Lo?c Hoguin wrote: > If anyone has setup PropEr tests in their open source project I'd really > like to take a look. Magnus Klaar changed the quickcheck suite of etorrent to use PropEr and we fire it from eunit at the moment. The etorrent_bcoding module has an example, I believe, https://github.com/jlouis/etorrent/blob/master/apps/etorrent/src/etorrent_bcoding.erl#L194 which is easy enough to understand. My own branch which I am working on is still using QuviQ Quickcheck Mini though. Old habits die hard. -- J. From ixmatus@REDACTED Sat Jul 23 20:48:25 2011 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Sat, 23 Jul 2011 11:48:25 -0700 Subject: [erlang-questions] Process pool map/3 implementation In-Reply-To: <45993791.129661311379846617.JavaMail.root@zimbra> References: <45993791.129661311379846617.JavaMail.root@zimbra> Message-ID: Robert, In my ppool implementation I re-create them, I don't recycle. I call it a "pool" because instead of breaking off "hunks" (sublists) then spawning x processes for each item in the sublist - if one process takes longer than the others, those resources sit idle till that one is done, then the subdivision process starts again etc... The "pool" implementation lets me delegate in a round robin fashion, I don't recycle, I create new processes as other processes finish to keep at most x number of processes working until the job is done. pmap in my use case would be "bad". Very bad. 3000 items in a list with about 1400 of those items making > 20 HTTP requests (the rest doing about 3 to 4 requests) would completely tank the machine and would also be irresponsible crawling. But, I *do* want to do the work in parallel - just not at that scale; so using a process pool strategy I limit the number of concurrent crawl workers to about 5 or 6; which is effective on the machine. On Fri, Jul 22, 2011 at 5:10 PM, Robert Virding < robert.virding@REDACTED> wrote: > But a pmap SHOULD start processes for all the elements in the list in > parallel. It is after all a 'P'map. In which case all the processes will be > running and processing in parallel as you want. The only reason I can see > for using a worker pool is if you actually want to LIMIT the number of > processes running at the same time. > > IMAO in Erlang there are only two reasons for using worker/process pools: > > - you want/need to limit the number of "things" running in parallel > - you actually do want to reuse a process for another computation, there is > something in the application which mandates reusing processes. > > Otherwise it is just extra work, process creation/termination is so fast > that there is no real gain in keeping them around to reuse. > > > Robert > > ----- "Parnell Springmeyer" wro te: > > Because the list has about 3000 items in it, and for each item about > 20-50 HTTP requests are made; I needed a way of parallelizing the operations > (instead of stepping through the list one by one) but in a controlled > fashion and using a round robin strategy (worker pool). > > > > > On Fri, Jul 22, 2011 at 6:10 AM, David Mercer wrote: > > >> >> I was curious about that, too. Hoping you'll get a response... >> > >> > > -----Original Message----- >> > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions- >> > > bounces@REDACTED] On Behalf Of Robert Virding >> > > Sent: Wednesday, July 20, 2011 8:42 PM >> > > To: Parnell Springmeyer >> > > Cc: erlang-questions >> > > Subject: Re: [erlang-questions] Process pool map/3 implementation >> > > >> > > One quick question: what was wrong with the straightforward solution >> of >> > > just spawning one process for each element in the list? Did this break >> > > or do you actually need more control? >> > > >> > > Robert >> > >> > > >> > > ----- "Parnell Springmeyer" wrote: >> > > >> > > > -----BEGIN PGP SIGNED MESSAGE----- >> > > > Hash: SHA1 >> > > > >> > > > For a work project I have a large list (thousands of items) to >> > > > process >> > > > and at first built a "pmap" implementation as per Joe's book until I >> > > > found the plists module (which is awesome btw). >> > > > >> > > > There is one glaring issue with the list -> subdivide -> spawn x >> > > > processes for n sublist items strategy; if an item in the sublist >> > > > takes >> > > > longer than all the other items it blocks the entire resource >> > > > allotment >> > > > until it is done. >> > > > >> > > > In most cases, the plists/pmap implementation works just fine >> because >> > > > the items in the list probably don't take more than a few >> > > > milliseconds >> > > > to map the fun over. However, it does become an issue when that is >> > > > not >> > > > the case. >> > > > >> > > > So, I figured the next best strategy would be to implement a process >> > > > pool since it would allow for slow running processes to continue >> > > > their >> > > > work while finished processes can die and new processes spawned into >> > > > the >> > > > pool ready for work - so none of the resources are sitting idle. >> > > > >> > > > Right now, my module isn't nearly as feature-complete as the plists >> > > > module is - this is only a drop in replacement for map. Please >> submit >> > > > your criticisms and comments to me at this address. >> > > > >> > > > You may find the code on BitBucket: >> > > > https://bitbucket.org/ixmatus/ppool >> > > > >> > > > - -- >> > > > Parnell "ixmatus" Springmeyer (http://ixmat.us) >> > > > -----BEGIN PGP SIGNATURE----- >> > > > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) >> > > > Comment: GPGTools - http://gpgtools.org >> > > > >> > > > iQEcBAEBAgAGBQJOJmKwAAoJEPvtlbpI1POL+asIAKPcR0SOw67hFwwIbmkf89sS >> > > > 4+Zx9hx1V/+86OVtXcqcOY+yxNcHezNEKkw8z2XHmDAWbeOl3bbINFySRXbQVydV >> > > > 854lArqCHRG+ZlJ6ZrgecXKf9mG8ldbK1InwEZWOVZBj63rhmloMaGiyTzmxA88S >> > > > 7mDNS4uhhpvRT2znpnsWt1x12IAzeayV0hf5/BLjp+b5FMZPc9oSa4n5uzyA9AVW >> > > > +av6hyuFfK32lhxUb4u3bVMaHOf2n/YwJexS25+NODcpkI3BLXNkrmKwgz8Lv/sA >> > > > omKzKTiuhpa0vTM+TLI9pn82GCJLdD+ON9DDOFN4ww+BnmXjhykiicBQCg7yhtQ= >> > > > =GP7K >> > > > -----END PGP SIGNATURE----- >> > >> > > > _______________________________________________ >> > > > erlang-questions mailing list >> > > > erlang-questions@REDACTED >> > > > http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ >> > > erlang-questions mailing list >> > > erlang-questions@REDACTED >> > > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > > > > > > -- > > Parnell "ixmatus" Springmeyer (http://ixmat.us) > > > > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Sat Jul 23 22:10:06 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sat, 23 Jul 2011 22:10:06 +0200 Subject: [erlang-questions] SMP enabled or not? Message-ID: Hi, Is there an Erlang way which let me know if: 1. the Erlang VM was compiled with SMP support or not? 2. the Erlang VM was started with SMP enabled (i.e "erl -smp enable" ) 3. how many cores are available on my machine? Regards Z. From b.nicholson@REDACTED Sat Jul 23 22:33:59 2011 From: b.nicholson@REDACTED (Barry Nicholson) Date: Sat, 23 Jul 2011 15:33:59 -0500 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: References: Message-ID: <4E2B3037.1050303@niceng.com> I have no idea about questions 1 & 2. Theoretically, you can use the following code to discover how many cpus are on a machine. application:start(sasl). application:start(os_mon). cpu_sup:util([per_cpu]). However, it doesn't work on Macs or at least mine. We also write a small library routine that parses /proc/cpuinfo on Linux. /proc/cpuinfo has a line of the form processor: x where x is the cpu id. We simply retrieved the largest x found in the file. Thus we then have the number of cpus on that machine. Remember this doesn't work on Windows, Mac, embedded or many other OSs. But the second method isn't an Erlang way. Barry Nicholson On 7/23/11 3:10 PM, Zabrane Mickael wrote: > Hi, > > Is there an Erlang way which let me know if: > > 1. the Erlang VM was compiled with SMP support or not? > 2. the Erlang VM was started with SMP enabled (i.e "erl -smp enable" ) > 3. how many cores are available on my machine? > > Regards > Z. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From grimlog@REDACTED Sat Jul 23 22:48:08 2011 From: grimlog@REDACTED (Michael Schreckenbauer) Date: Sat, 23 Jul 2011 22:48:08 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: References: Message-ID: <1972053.cALKWsmFmD@pc> Hi, On Saturday 23 July 2011 22:10:06 Zabrane Mickael wrote: > Hi, > Is there an Erlang way which let me know if: > 1. the Erlang VM was compiled with SMP support or not? > 2. the Erlang VM was started with SMP enabled (i.e "erl -smp enable" ) erlang:system_info(smp_support). > 3. how many cores are available on my machine? erlang:system_info(logical_processors_available). > Regards > Z. HTH, Michael From erlangy@REDACTED Sat Jul 23 22:56:31 2011 From: erlangy@REDACTED (erlang) Date: Sat, 23 Jul 2011 22:56:31 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: <1972053.cALKWsmFmD@pc> References: <1972053.cALKWsmFmD@pc> Message-ID: Thanks guys. Le 23 juil. 2011 ? 22:48, Michael Schreckenbauer a ?crit : > Hi, > > On Saturday 23 July 2011 22:10:06 Zabrane Mickael wrote: >> Hi, > >> Is there an Erlang way which let me know if: > >> 1. the Erlang VM was compiled with SMP support or not? >> 2. the Erlang VM was started with SMP enabled (i.e "erl -smp enable" ) > > erlang:system_info(smp_support). > >> 3. how many cores are available on my machine? > > erlang:system_info(logical_processors_available). From frgo@REDACTED Sun Jul 24 12:00:12 2011 From: frgo@REDACTED (Frank Goenninger) Date: Sun, 24 Jul 2011 12:00:12 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: <1972053.cALKWsmFmD@pc> References: <1972053.cALKWsmFmD@pc> Message-ID: <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> Am 23.07.2011 um 22:48 schrieb Michael Schreckenbauer: > erlang:system_info(smp_support). > > erlang:system_info(logical_processors_available). $ erl Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> erlang:system_info(smp_support). true 2> erlang:system_info(logical_processors_available). unknown ... on my 4 core MacBook Pro ... Cheers Frank From ponton@REDACTED Sun Jul 24 12:12:09 2011 From: ponton@REDACTED (Tomasz Maciejewski) Date: Sun, 24 Jul 2011 12:12:09 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> References: <1972053.cALKWsmFmD@pc> <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> Message-ID: Dnia 24-07-2011 o 12:00:12 Frank Goenninger napisa?(a): > [smp:4:4] Here it is. -- Tomasz Maciejewski From frgo@REDACTED Sun Jul 24 12:13:52 2011 From: frgo@REDACTED (Frank Goenninger) Date: Sun, 24 Jul 2011 12:13:52 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: References: <1972053.cALKWsmFmD@pc> <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> Message-ID: <79D1AC4D-8C1C-4375-908F-A626FA689A75@me.com> Am 24.07.2011 um 12:12 schrieb Tomasz Maciejewski: > Dnia 24-07-2011 o 12:00:12 Frank Goenninger napisa?(a): > >> [smp:4:4] > > Here it is. I know. But why does this show up as "unknown" when asked for programmatically ? Frank From sam@REDACTED Sun Jul 24 12:42:43 2011 From: sam@REDACTED (Sam Elliott) Date: Sun, 24 Jul 2011 11:42:43 +0100 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: <79D1AC4D-8C1C-4375-908F-A626FA689A75@me.com> References: <1972053.cALKWsmFmD@pc> <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> <79D1AC4D-8C1C-4375-908F-A626FA689A75@me.com> Message-ID: try 1> erlang:system_info(logical_processors). 2 That got me the right thing on my mbp when logical_processors_available wasn't. Maybe also try: 2> erlang:system_info(logical_processors_online). 2 Sam -- Sam Elliott sam@REDACTED -- On Sun, Jul 24, 2011 at 11:13 AM, Frank Goenninger wrote: > > Am 24.07.2011 um 12:12 schrieb Tomasz Maciejewski: > > > Dnia 24-07-2011 o 12:00:12 Frank Goenninger napisa?(a): > > > >> [smp:4:4] > > > > Here it is. > > I know. But why does this show up as "unknown" when asked for > programmatically ? > > Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frgo@REDACTED Sun Jul 24 13:40:37 2011 From: frgo@REDACTED (Frank Goenninger) Date: Sun, 24 Jul 2011 13:40:37 +0200 Subject: [erlang-questions] SMP enabled or not? In-Reply-To: References: <1972053.cALKWsmFmD@pc> <8A6E3773-5DF1-4D2B-9A77-C1D7EC0BFCCD@me.com> <79D1AC4D-8C1C-4375-908F-A626FA689A75@me.com> Message-ID: <7DA204C2-55F6-4DD8-8DEC-D315D0F30ED6@me.com> Am 24.07.2011 um 12:42 schrieb Sam Elliott: > try > > 1> erlang:system_info(logical_processors). > 2 > > That got me the right thing on my mbp when logical_processors_available wasn't. Maybe also try: > > 2> erlang:system_info(logical_processors_online). > 2 Cool! Works! Thanks. Frank From s.j.thompson@REDACTED Sun Jul 24 22:24:51 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Sun, 24 Jul 2011 21:24:51 +0100 Subject: [erlang-questions] Fwd: Call for Submissions for a Special Issue of ACM TOCE References: <4E2BEA46.7030008@kent.ac.uk> Message-ID: <1BD487FB-168C-47BF-8080-CF06745AE309@kent.ac.uk> Call for Submissions for a Special Issue of ACM Transactions on Computing Education on Concurrent, Parallel and Distributed Computation The past few years have seen a significant change in the computer architecture that is readily available to students and educators: for example, multicore processors and cloud computing. Furthermore, programming languages used for teaching have multithreading defined within the language. Even visual programming environments for young people like Scratch and Alice support concurrency. The editors of this Special Issue are looking for papers describing innovative approaches to teaching and learning this subject. Preference will be given to papers that include an empirical evaluation. A preliminary one-page abstract of the paper should be sent to the editors of the Special Issue by 1 October 2011. The editors will assess the relevance of the papers for the Special Issue by 15 October. The deadline for submission of full papers is 1 December 2011. Submissions must be done via Manuscript Central (http://mc.manuscriptcentral.com/toce). In the cover letter, indicate that the paper is for the Special Issue on Concurrent, Parallel and Distributed Computation. See the TOCE review criteria at see http://toce.acm.org/authors.html). Guest editors: Moti Ben-Ari, Weizmann Institute of Science, moti.ben-ari@REDACTED Dan Garcia, University of California Berkeley, ddgarcia@REDACTED Tom Murphy, Contra Costa College, tmurphy@REDACTED > ############################ Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From jason.bosco@REDACTED Sun Jul 24 22:30:18 2011 From: jason.bosco@REDACTED (Jason Bosco) Date: Sun, 24 Jul 2011 13:30:18 -0700 Subject: [erlang-questions] compiling problems under OS X Lion Message-ID: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> This didn't work for me. I tried CFLAGS=-O0 ./configure --enable-darwin-64bit and then make but it stopped with this error: ld: warning: ignoring file /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/zlib/obj/i386-apple-darwin11.0.0/opt/libz.a, file was built for archive which is not the architecture being linked (x86_64) ld: warning: ignoring file /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/pcre/obj/i386-apple-darwin11.0.0/opt/libepcre.a, file was built for archive which is not the architecture being linked (x86_64) Undefined symbols for architecture x86_64:... Any ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sun Jul 24 23:53:30 2011 From: tony@REDACTED (Tony Rogvall) Date: Sun, 24 Jul 2011 23:53:30 +0200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: <1783425682.129231311358461747.JavaMail.root@zimbra> <20229112.129251311358554198.JavaMail.root@zimbra> Message-ID: On 22 jul 2011, at 20:35, Tristan Sloughter wrote: > I think everyone has suggested let bindings at some point in time for Erlang :) > 'let' has been a 'reserved' word since at least 1998. I could not find any trace of it in the sources from 1994. Maybe robert knows more about when this was first planned for ? And why it was never introduced :-) /Tony > On Fri, Jul 22, 2011 at 1:23 PM, Tim Watson wrote: > > I very much like the idea of a proper let (or a where) and I think it would make variable handling clearer. Variables are handled in a consistent way but it is not always easy to work out what should happen. Most people never ave problems with this as most people write pretty "simple" code (from the compiler point of view). Adding local functions, which I think would be a Good Thing, in the same way as variables are handled today would be very confusing so some form of 'let' construct would be necessary, at least for them. Then we might as well have for variables as well. > > > > I like the idea of let bindings - I think someone has suggested this > before. It might have been Richard, although apologies if I've got > that wrong. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Have run Make so many times I dunno what's installed anymore" -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Mon Jul 25 00:02:57 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 24 Jul 2011 23:02:57 +0100 (BST) Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: Message-ID: I can't remember when it was added. We/some-of-use/me thought it wood be a Good Thing so I added it to the reserved words, although we never got around to adding it. Not everyone was convinced. For info 'cond' is also a reserved word, another good idea we never got around to adding. :-) I have an extended 'cond' in LFE (can do pattern matching as a test) and it is useful. Robert ----- Original Message ----- > On 22 jul 2011, at 20:35, Tristan Sloughter wrote: > > I think everyone has suggested let bindings at some point in time > > for > > Erlang :) > > 'let' has been a 'reserved' word since at least 1998. I could not > find any trace of it in the sources from 1994. > Maybe robert knows more about when this was first planned for ? And > why it was never introduced :-) > /Tony > > On Fri, Jul 22, 2011 at 1:23 PM, Tim Watson < > > watson.timothy@REDACTED > wrote: > > > > > I very much like the idea of a proper let (or a where) and I > > > > think > > > > it would make variable handling clearer. Variables are handled > > > > in > > > > a consistent way but it is not always easy to work out what > > > > should > > > > happen. Most people never ave problems with this as most people > > > > write pretty "simple" code (from the compiler point of view). > > > > Adding local functions, which I think would be a Good Thing, in > > > > the same way as variables are handled today would be very > > > > confusing so some form of 'let' construct would be necessary, > > > > at > > > > least for them. Then we might as well have for variables as > > > > well. > > > > > > > > > > > > > I like the idea of let bindings - I think someone has suggested > > > this > > > > > > before. It might have been Richard, although apologies if I've > > > got > > > > > > that wrong. > > > > > > _______________________________________________ > > > > > > erlang-questions mailing list > > > > > > erlang-questions@REDACTED > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > "Have run Make so many times I dunno what's installed anymore" > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon Jul 25 02:54:58 2011 From: sean@REDACTED (Sean Moss-Pultz) Date: Mon, 25 Jul 2011 08:54:58 +0800 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> References: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> Message-ID: On Mon, Jul 25, 2011 at 4:30 AM, Jason Bosco wrote: > > This didn't work for me. I?tried CFLAGS=-O0 ./configure --enable-darwin-64bit and then make but it stopped with this error: > > ld: warning: ignoring file /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/zlib/obj/i386-apple-darwin11.0.0/opt/libz.a, file was built for archive which is not the architecture being linked (x86_64) ld: warning: ignoring file /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/pcre/obj/i386-apple-darwin11.0.0/opt/libepcre.a, file was built for archive which is not the architecture being linked (x86_64) Undefined symbols for architecture x86_64:... > > Any ideas? Jason I don't know but I was able to build by using Xcode (4.1) and that command on Lion. Actually I wanted SSL so I added --with-ssl. Did you try building before you added those flags? You might need to clean out the source directory first. Sean From jason.bosco@REDACTED Mon Jul 25 05:32:08 2011 From: jason.bosco@REDACTED (Jason Bosco) Date: Sun, 24 Jul 2011 20:32:08 -0700 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> Message-ID: Sean, I did a "make clean" before running CFLAGS=-O0 ./configure --enable-darwin-64bit once again. That should "clean out the source directory" right? Jason. On Sun, Jul 24, 2011 at 5:54 PM, Sean Moss-Pultz wrote: > On Mon, Jul 25, 2011 at 4:30 AM, Jason Bosco > wrote: > > > > This didn't work for me. I tried CFLAGS=-O0 ./configure > --enable-darwin-64bit and then make but it stopped with this error: > > > > ld: warning: ignoring file > /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/zlib/obj/i386-apple-darwin11.0.0/opt/libz.a, > file was built for archive which is not the architecture being linked > (x86_64) ld: warning: ignoring file > /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/pcre/obj/i386-apple-darwin11.0.0/opt/libepcre.a, > file was built for archive which is not the architecture being linked > (x86_64) Undefined symbols for architecture x86_64:... > > > > Any ideas? > > Jason > > I don't know but I was able to build by using Xcode (4.1) and that > command on Lion. Actually I wanted SSL so I added --with-ssl. Did you > try building before you added those flags? You might need to clean out > the source directory first. > > Sean > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Mon Jul 25 06:36:46 2011 From: daniel@REDACTED (Daniel Luna) Date: Mon, 25 Jul 2011 04:36:46 +0000 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> Message-ID: It should. However, it doesn't. Has been broken for years. The makefiles for Erlang/OTP are not particularly good. Hasn't bothered me enough to fix though since you normally only build once. Cheers, Daniel On 24 Jul 2011 23:32, "Jason Bosco" wrote: > Sean, > > I did a "make clean" before running CFLAGS=-O0 ./configure > --enable-darwin-64bit once again. That should "clean out the source > directory" right? > > Jason. > > On Sun, Jul 24, 2011 at 5:54 PM, Sean Moss-Pultz wrote: > >> On Mon, Jul 25, 2011 at 4:30 AM, Jason Bosco >> wrote: >> > >> > This didn't work for me. I tried CFLAGS=-O0 ./configure >> --enable-darwin-64bit and then make but it stopped with this error: >> > >> > ld: warning: ignoring file >> /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/zlib/obj/i386-apple-darwin11.0.0/opt/libz.a, >> file was built for archive which is not the architecture being linked >> (x86_64) ld: warning: ignoring file >> /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/pcre/obj/i386-apple-darwin11.0.0/opt/libepcre.a, >> file was built for archive which is not the architecture being linked >> (x86_64) Undefined symbols for architecture x86_64:... >> > >> > Any ideas? >> >> Jason >> >> I don't know but I was able to build by using Xcode (4.1) and that >> command on Lion. Actually I wanted SSL so I added --with-ssl. Did you >> try building before you added those flags? You might need to clean out >> the source directory first. >> >> Sean >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.bosco@REDACTED Mon Jul 25 07:08:50 2011 From: jason.bosco@REDACTED (Jason Bosco) Date: Sun, 24 Jul 2011 22:08:50 -0700 Subject: [erlang-questions] compiling problems under OS X Lion In-Reply-To: References: <757E433A-976D-48ED-AE03-29E76759C6A8@gmail.com> Message-ID: Daniel, I ran 'CFLAGS=-O0 ./configure --enable-darwin-64bit' and then 'make' on a fresh copy of the source and it compiled fine. Thanks! Regards, Jason. On Sun, Jul 24, 2011 at 9:36 PM, Daniel Luna wrote: > It should. > > However, it doesn't. Has been broken for years. > > The makefiles for Erlang/OTP are not particularly good. Hasn't bothered me > enough to fix though since you normally only build once. > > Cheers, > > Daniel > On 24 Jul 2011 23:32, "Jason Bosco" wrote: > > Sean, > > > > I did a "make clean" before running CFLAGS=-O0 ./configure > > --enable-darwin-64bit once again. That should "clean out the source > > directory" right? > > > > Jason. > > > > On Sun, Jul 24, 2011 at 5:54 PM, Sean Moss-Pultz > wrote: > > > >> On Mon, Jul 25, 2011 at 4:30 AM, Jason Bosco > >> wrote: > >> > > >> > This didn't work for me. I tried CFLAGS=-O0 ./configure > >> --enable-darwin-64bit and then make but it stopped with this error: > >> > > >> > ld: warning: ignoring file > >> > /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/zlib/obj/i386-apple-darwin11.0.0/opt/libz.a, > >> file was built for archive which is not the architecture being linked > >> (x86_64) ld: warning: ignoring file > >> > /Users/abcd/Downloads/otp_src_R14B03/erts/emulator/pcre/obj/i386-apple-darwin11.0.0/opt/libepcre.a, > >> file was built for archive which is not the architecture being linked > >> (x86_64) Undefined symbols for architecture x86_64:... > >> > > >> > Any ideas? > >> > >> Jason > >> > >> I don't know but I was able to build by using Xcode (4.1) and that > >> command on Lion. Actually I wanted SSL so I added --with-ssl. Did you > >> try building before you added those flags? You might need to clean out > >> the source directory first. > >> > >> Sean > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Mon Jul 25 14:19:15 2011 From: paulperegud@REDACTED (=?UTF-8?Q?Pawe=C5=82_Peregud?=) Date: Mon, 25 Jul 2011 14:19:15 +0200 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> Message-ID: I wonder how did you solve the issue with eunit hiding output from PropEr? I do following on test setup: meck:new(io, [unstick, passthrough]), meck:expect(io, fwrite, fun (FString, Args) -> meck:passthrough([user, FString, Args]) end) This approach is ugly and also very slow. Does anyone have a better solution? Best, PP. 2011/6/24 Roman Shestakov > thanks a lot Bob, this is very helpful > Regards, Roman > > ------------------------------ > *From:* Bob Ippolito > *To:* Fr?d?ric Trottier-H?bert > *Cc:* Roman Shestakov ; > erlang-questions@REDACTED > *Sent:* Fri, 24 June, 2011 18:05:37 > *Subject:* Re: [erlang-questions] how to run PropEr tests from Eunit? > > This is what I've been using: > > proper_test_() -> > [{atom_to_list(F), > fun () -> ?assert(proper:quickcheck(?MODULE:F(), [long_result])) end} > || {F, 0} <- ?MODULE:module_info(exports), F > 'prop_', F < 'prop`']. > > 2011/6/23 Fr?d?ric Trottier-H?bert : > > The simplest way I know to do them individually is with separate tests: > > proper_test_() -> > > [?_assert(proper:quickcheck(prop_some_property())), > > ?_assert(proper:quickcheck(prop_some_property())), > > ...]. > > You can then create macros to do it for you of the form > > -define(PROPTEST(A), ?_assert(proper:quickcheck(A()))). > > and then call > > proper_test_() -> > > [?PROPTEST(prop_something), ?PROPTEST(prop_something_else), > > ?PROPTEST(prop_other), ?PROPTEST(prop_last)]. > > or whatever you feel like. > > If you want to run them all, make sure your module's properties all start > > with the prefix 'prop_'. Then you can do something like > > proper_test() -> > > [] = proper:module(?MODULE). > > which will find all the properties of the current module and run them for > > you. Also, don't forget to always include the proper include file first, > > because both PropEr and EUnit define a LET macro, but the PropEr one is > the > > only one I've seen actively used. > > > > -- > > Fred H?bert > > http://www.erlang-solutions.com > > > > > > On 2011-06-23, at 18:12 PM, Roman Shestakov wrote: > > > > hello, > > > > I am trying to run PropEr tests from Eunit and I must admit, I can't > figure > > out how to run them > > > > > > lets say I want to test the following three functions > > > > %%-------------------------------------------------------------------- > > %% @doc > > %% convert Date to atom > > %% @end > > %%-------------------------------------------------------------------- > > -spec date_to_atom(date()) -> atom(). > > date_to_atom({Year, Month, Day}) -> > > list_to_atom(date_to_string({Year, Month, Day})). > > > > %%-------------------------------------------------------------------- > > %% @doc > > %% Convert Date to string > > %% @end > > %%-------------------------------------------------------------------- > > -spec date_to_string(date()) -> string(). > > date_to_string({Year, Month, Day}) -> > > lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, > > Day])). > > > > > > %%-------------------------------------------------------------------- > > %% @doc > > %% convert Date represented as atom into Erlang date format > > %% @end > > %%-------------------------------------------------------------------- > > -spec atom_to_date(atom()) -> date(). > > atom_to_date(Date) -> > > Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), > > Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), > > Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), > > {Year, Month, Day}. > > > > > > > > with the following property: > > > > proper_time_to_atom() -> > > ?FORALL(Date, date(), > > begin > > EncDate = > ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), > > EncDate =:= Date > > end). > > > > running the following from erl works fine > > proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). > > > > > > but how do I make eunit to run these test? > > > > > > tried this , but it doesn't seem to work > > > > time_to_atom_test_() -> > > ?_test(proper:quickcheck(proper_time_to_atom())). > > > > > > any ideas? > > > > Regards, Roman > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Jul 25 14:59:22 2011 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 25 Jul 2011 08:59:22 -0400 Subject: [erlang-questions] how to run PropEr tests from Eunit? In-Reply-To: References: <1308867174.83770.YahooMailRC@web25402.mail.ukl.yahoo.com> <93BD232D-C14F-4B16-80D2-96C24EE44BB3@erlang-solutions.com> <1308939214.73194.YahooMailRC@web25404.mail.ukl.yahoo.com> Message-ID: You can just change the process' group leader back to the 'user' process (or any other process depending on your app) from within the test, and then set it back to what it was before using erlang:group_leader/0 and erlang:group_leader/2. On Mon, Jul 25, 2011 at 8:19 AM, Pawe? Peregud wrote: > I wonder how did you solve the issue with eunit hiding output from PropEr? > > I do following on test setup: > > meck:new(io, [unstick, passthrough]), > meck:expect(io, fwrite, fun > (FString, Args) -> meck:passthrough([user, > FString, Args]) > end) > > This approach is ugly and also very slow. Does anyone have a better > solution? > > Best, > PP. > > > > > 2011/6/24 Roman Shestakov > >> thanks a lot Bob, this is very helpful >> Regards, Roman >> >> ------------------------------ >> *From:* Bob Ippolito >> *To:* Fr?d?ric Trottier-H?bert >> *Cc:* Roman Shestakov ; >> erlang-questions@REDACTED >> *Sent:* Fri, 24 June, 2011 18:05:37 >> *Subject:* Re: [erlang-questions] how to run PropEr tests from Eunit? >> >> This is what I've been using: >> >> proper_test_() -> >> [{atom_to_list(F), >> fun () -> ?assert(proper:quickcheck(?MODULE:F(), [long_result])) >> end} >> || {F, 0} <- ?MODULE:module_info(exports), F > 'prop_', F < 'prop`']. >> >> 2011/6/23 Fr?d?ric Trottier-H?bert : >> > The simplest way I know to do them individually is with separate tests: >> > proper_test_() -> >> > [?_assert(proper:quickcheck(prop_some_property())), >> > ?_assert(proper:quickcheck(prop_some_property())), >> > ...]. >> > You can then create macros to do it for you of the form >> > -define(PROPTEST(A), ?_assert(proper:quickcheck(A()))). >> > and then call >> > proper_test_() -> >> > [?PROPTEST(prop_something), ?PROPTEST(prop_something_else), >> > ?PROPTEST(prop_other), ?PROPTEST(prop_last)]. >> > or whatever you feel like. >> > If you want to run them all, make sure your module's properties all >> start >> > with the prefix 'prop_'. Then you can do something like >> > proper_test() -> >> > [] = proper:module(?MODULE). >> > which will find all the properties of the current module and run them >> for >> > you. Also, don't forget to always include the proper include file first, >> > because both PropEr and EUnit define a LET macro, but the PropEr one is >> the >> > only one I've seen actively used. >> > >> > -- >> > Fred H?bert >> > http://www.erlang-solutions.com >> > >> > >> > On 2011-06-23, at 18:12 PM, Roman Shestakov wrote: >> > >> > hello, >> > >> > I am trying to run PropEr tests from Eunit and I must admit, I can't >> figure >> > out how to run them >> > >> > >> > lets say I want to test the following three functions >> > >> > %%-------------------------------------------------------------------- >> > %% @doc >> > %% convert Date to atom >> > %% @end >> > %%-------------------------------------------------------------------- >> > -spec date_to_atom(date()) -> atom(). >> > date_to_atom({Year, Month, Day}) -> >> > list_to_atom(date_to_string({Year, Month, Day})). >> > >> > %%-------------------------------------------------------------------- >> > %% @doc >> > %% Convert Date to string >> > %% @end >> > %%-------------------------------------------------------------------- >> > -spec date_to_string(date()) -> string(). >> > date_to_string({Year, Month, Day}) -> >> > lists:flatten(io_lib:format("~4.10.0B~2.10.0B~2.10.0B", [Year, Month, >> > Day])). >> > >> > >> > %%-------------------------------------------------------------------- >> > %% @doc >> > %% convert Date represented as atom into Erlang date format >> > %% @end >> > %%-------------------------------------------------------------------- >> > -spec atom_to_date(atom()) -> date(). >> > atom_to_date(Date) -> >> > Year = list_to_integer(lists:sublist(atom_to_list(Date),1,4)), >> > Month = list_to_integer(lists:sublist(atom_to_list(Date),5,2)), >> > Day = list_to_integer(lists:sublist(atom_to_list(Date),7,2)), >> > {Year, Month, Day}. >> > >> > >> > >> > with the following property: >> > >> > proper_time_to_atom() -> >> > ?FORALL(Date, date(), >> > begin >> > EncDate = >> ec_time_fns:atom_to_date(ec_time_fns:date_to_atom(Date)), >> > EncDate =:= Date >> > end). >> > >> > running the following from erl works fine >> > proper:quickcheck(ec_time_fns_tests:proper_time_to_atom()). >> > >> > >> > but how do I make eunit to run these test? >> > >> > >> > tried this , but it doesn't seem to work >> > >> > time_to_atom_test_() -> >> > ?_test(proper:quickcheck(proper_time_to_atom())). >> > >> > >> > any ideas? >> > >> > Regards, Roman >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Mon Jul 25 18:15:10 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Tue, 26 Jul 2011 00:15:10 +0800 Subject: [erlang-questions] weird http parsing Message-ID: Hi, I'm using inet:setopts(Sock, [{active,true},{packet, http}]) to receive http packets. I found that Erlang silently modifies some http header name fields from like "Sec-WebSocket-Origin" to "Sec-Websocket-Origin". But another fields "Sec-WebSocket-Version" is not modified. It's quite strange. I know http header field name is case insensitive but Erlang should not silently and inconsistently modify it. After I found it I tried to use all lower case. But "erlang:decode_packet" decodes some of the fields into atoms and other not-recognized ones to string so it's quite inconvenient to use all lower case. Any ideas to mitigate this? Thanks Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Mon Jul 25 19:22:39 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 25 Jul 2011 19:22:39 +0200 Subject: [erlang-questions] weird http parsing In-Reply-To: References: Message-ID: <4E2DA65F.8090105@erix.ericsson.se> Andy W. Song wrote: > Hi, > > I'm using inet:setopts(Sock, [{active,true},{packet, http}]) to receive http > packets. I found that Erlang silently modifies some http header name fields > from like "Sec-WebSocket-Origin" to "Sec-Websocket-Origin". But another > fields "Sec-WebSocket-Version" is not modified. It's quite strange. > > Yes, header names shorter than 21 characters are modified, while longer names are not. I agree it's a bit strange. > I know http header field name is case insensitive but Erlang should not > silently and inconsistently modify it. > Wouldn't the best behavior be to *consistently* modify the header names to a predefined format (uppercase only first and after '-'). Then you can do an exact string compare. > After I found it I tried to use all lower case. But "erlang:decode_packet" > decodes some of the fields into atoms and other not-recognized ones to > string so it's quite inconvenient to use all lower case. > > Yes, some header names are recognized and presented as atoms. That is documented in erlang:decode_packet. > Any ideas to mitigate this? > > If you have control over input, use uppercase only as first character and directly after '-'. /Sverker, Erlang/OTP From romanshestakov@REDACTED Mon Jul 25 20:34:23 2011 From: romanshestakov@REDACTED (Roman Shestakov) Date: Mon, 25 Jul 2011 19:34:23 +0100 (BST) Subject: [erlang-questions] problem with starting gproc Message-ID: <1311618863.26599.YahooMailRC@web25405.mail.ukl.yahoo.com> hello, does anybody else have a problem with gproc? I started having some issues since I upgraded to Lion, the fresh gproc build from git head wouldn't start. ( I rebuilt erlang install after macos 10.7 but it didn't help) Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> application:start(gproc). =INFO REPORT==== 25-Jul-2011::19:26:14 === application: gproc exited: {shutdown,{gproc_app,start,[normal,[]]}} type: temporary {error,{shutdown,{gproc_app,start,[normal,[]]}}} 2> any ideas what might be causing it ? I wander if it is my machine problem or generic issue. Regards, Roman -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikhail.sobolev@REDACTED Mon Jul 25 21:27:05 2011 From: mikhail.sobolev@REDACTED (Mikhail Sobolev) Date: Mon, 25 Jul 2011 22:27:05 +0300 Subject: [erlang-questions] Feedback on the code, please? :) Message-ID: Hi, I have written a simple tool for finding duplicate files in the given set of directories[1]. I'd appreciate if somebody could have a look and comment on the code :) -- Misha (learning erlang) [1] https://github.com/sa2ajj/dups/blob/master/dups From pmm@REDACTED Mon Jul 25 23:34:51 2011 From: pmm@REDACTED (Peter Mechlenborg) Date: Mon, 25 Jul 2011 23:34:51 +0200 Subject: [erlang-questions] Feedback on the code, please? :) In-Reply-To: References: Message-ID: Hi Mikhail One comment on your code. I always put the function clauses for the base-case at the top and I think this is the norm. Others will hopefully correct me if I am mistaken. Other than that I don't have much to comment. It looks good in my eyes. You can find a modified version of you code here: https://gist.github.com/1105252 Not sure if it is better than your version, though. Best regards, -- Peter Mechlenborg On Mon, Jul 25, 2011 at 9:27 PM, Mikhail Sobolev wrote: > Hi, > > I have written a simple tool for finding duplicate files in the given > set of directories[1]. I'd appreciate if somebody could have a look > and comment on the code :) > > -- > Misha (learning erlang) > > [1] https://github.com/sa2ajj/dups/blob/master/dups > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jul 26 05:24:39 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 26 Jul 2011 15:24:39 +1200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <3D59DDF8-D903-4FC2-BAF3-046C6B6108A1@cs.otago.ac.nz> There was once a proposal to have begin within end where the would be visible within the but not outside. begin...end survives in the language, but 'within' never made it. From wsongcn@REDACTED Tue Jul 26 08:13:12 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Tue, 26 Jul 2011 14:13:12 +0800 Subject: [erlang-questions] How to differentiate info_msg and error_msg Message-ID: Hi, I can print messages to error log using error_logger:info_msg and error_logger:error_msg. But I can't find a way to enable only error_msg while mask info_msg, just like log level in a lot of Linux software. Any idea? Thanks Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji.rikitake@REDACTED Tue Jul 26 08:47:44 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Tue, 26 Jul 2011 15:47:44 +0900 Subject: [erlang-questions] Fwd: Erlang Workshop 2011 / ICFP 2011 registration site is now open In-Reply-To: References: Message-ID: (My apologies if you receive this for multiple times.) Hi, I'd like to announce that ICFP 2011 registration site (also for Erlang Workshop 2011) is now open. For Erlang Workshop 2011, the price on or before 15-AUG-2011 is USD110 for students, USD180 for ACM/SIGPLAN members, USD220 for the non-members. The on-site price after 15-AUG-2011 will be students: USD130, ACM/SIGPLAN members: USD220, ACM/SIGPLAN non-members: USD260. For the further details, check out the following URLs: https://regmaster3.com/2011conf/ICFP11/register.php https://regmaster3.com/2011conf/ICFP11/register.pdf Regards, Kenji Rikitake, Erlang Workshop 2011 General Chair From anupam.kapoor@REDACTED Tue Jul 26 09:36:05 2011 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Tue, 26 Jul 2011 13:06:05 +0530 Subject: [erlang-questions] encoding packets a style+efficiency question Message-ID: hi all, i have been playing with an implementation of tftp (rfc-1350). encoding/decoding tftp-pdu's is trivial with erlang's bit syntax. there seem to be couple of approaches to encoding/decoding tftp-pdu's: 1. use native "in-place" encode/decode 2. have a 'generic' encode/decode routine which accepts - a list of pdu primitives e.g. 'byte'/'string' etc. and - a list of corresponding values and returns the binary packet. for example, encoding a tftp-data packet looks like this: ,---- | %% encode data packet | encode_data_pkt(BlkNum, Data) -> | encode_packet([?BYTE, ?BYTE, ?RAW], [?TFTP_OPCODE_DATA, BlkNum, Data]). `---- to me, approach #2 seems pretty close to rfc's packet-format specification. for example, the data-packet is shown as 2 bytes 2 bytes n bytes ---------------------------------- | Opcode | Block # | Data | ---------------------------------- can you please provide your suggestion on whether #2 might be a good approach to follow for complicated protocols e.g. gtp-v2 which defines a large number of messages ? also, are there better (increasing order of readability, performance etc.) approaches for doing this ? thank you kind regards anupam ps: the attached file contains encoders for tftp messages... -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. -------------- next part -------------- A non-text attachment was scrubbed... Name: misc.erl Type: text/x-erlang Size: 2269 bytes Desc: not available URL: From wiener.guy@REDACTED Tue Jul 26 09:39:59 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Tue, 26 Jul 2011 10:39:59 +0300 Subject: [erlang-questions] callcc or CPS in Erlang? Message-ID: Hello everyone, Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? Alt., is there some what to encourage Continuation-Passing Style programming in Erlang, other than just recommending that all consequent function calls will be in tail-call position? Thanks, Guy -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Jul 26 09:49:52 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 26 Jul 2011 09:49:52 +0200 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: <3D59DDF8-D903-4FC2-BAF3-046C6B6108A1@cs.otago.ac.nz> References: <3D59DDF8-D903-4FC2-BAF3-046C6B6108A1@cs.otago.ac.nz> Message-ID: "let" I can do without - I've never thought this to be a problem (and I don't want to re-iterate the pros and cons here). "letrec" on the other hand I find sadly missing. Choosing names for auxiliary functions ("helper" functions) is painful. Suppose I want to define foo/2 and bar/3 which are mutually recursive but which also call a number of private auxiliary functions. The only encapsulation mechanism available is the module. I want to say: letrec foo(X,Y) -> ... foo_1(...) bar(A,B,C) -> ... foo-1(...) bar_26(...) where foo_1(...) -> bar_26(...) -> end. Only the functions foo/2 and bar/3 would be visible from outside the letrec. The is already a mechanism for (almost) this. The module. But there are two problems with modules - we can't in-line them and we can't have "anonymous" modules. With a small sleight of hand we can transform letrecs into a very Erlangly syntax. -module(). -export[foo/2,bar/2]). %% definitions of foo/2, bar/2 and anything else they call -end(). Adding this to the language is a trivial exercise in pre-processing. This would incidentally solve some other problems do do with the scope of comments of macros. Analyzing source code is difficult because there is no notion of the scope of a comment or the scope of a macro. Comments are not attached to any particular function and so on. This makes refactoring tools and IDEs messy to say the least. On Tue, Jul 26, 2011 at 5:24 AM, Richard O'Keefe wrote: > There was once a proposal to have > ? ? ? ?begin > ? ? ? ? ? ? > ? ? ? ?within > ? ? ? ? ? ? > ? ? ? ?end > where the would be visible within the but not > outside. ?begin...end survives in the language, but 'within' never made it. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From carlsson.richard@REDACTED Tue Jul 26 10:47:28 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 26 Jul 2011 11:47:28 +0300 Subject: [erlang-questions] Feedback on the code, please? :) In-Reply-To: References: Message-ID: <4E2E7F20.9050505@gmail.com> On 07/26/2011 12:34 AM, Peter Mechlenborg wrote: > One comment on your code. I always put the function clauses for the > base-case at the top and I think this is the norm. Others will > hopefully correct me if I am mistaken. Efficiency-wise it's better to leave the base cases last. (If the clauses may safely be reordered, the compiler might fix this for you, but you can't rely on it in general.) Clauses are tried in order, so if you loop over a list of N elements, the base case will be true only in the last case, and in all the other N-1 cases, you will waste a little time checking "is this an empty list... nope... is it a nonempty list... yes - OK, let's run the second clause". The exception is if you only test in the base case and have a catch-all clause (i.e., without any test) for the recursive case; e.g., "is X zero... no... OK, let's run the second clause". And if you find that putting the base cases last makes them end up too far down on the page, and too far away from the head of the recursive clause, it's a sign that your clause body for the recursive case is way too long and should be broken out into a separate function. /Richard From icfp.publicity@REDACTED Tue Jul 26 10:58:33 2011 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Tue, 26 Jul 2011 10:58:33 +0200 Subject: [erlang-questions] ICFP 2011: Call for participation Message-ID: ===================================================================== Call for Participation The 16th ACM SIGPLAN International Conference on Functional Programming (ICFP 2011) http://www.icfpconference.org/icfp2011/ Tokyo, Japan September 19-21, 2011 ===================================================================== ICFP provides a forum for researchers and developers to hear about the latest work on the design, implementations, principles, and uses of functional programming. The conference covers the entire spectrum of work, from practice to theory, including its peripheries. * Program: http://www.icfpconference.org/icfp2011/program.html * Registration link: https://regmaster3.com/2011conf/ICFP11/register.php * Local arrangements (including travel and accommodation): http://www.biglab.org/icfp11local/index.html Schedule including related events: September 18 Workshop on Generic Programming (WGP) Workshop on High-Level Parallel Programming and Applications (HLPP) Workshop on ML September 19-21 ICFP September 22 Commercial Users of Functional Programming ? Day 1 (CUFP Tutorials) Haskell Symposium September 23 Commercial Users of Functional Programming ? Day 2 (CUFP Tutorials) Erlang Workshop Haskell Implementors' Workshop September 24 Commercial Users of Functional Programming ? Day 3 (CUFP Talks) Continuation Workshop Conference organizers: * General Co-Chairs: Manuel Chakravarty, University of New South Wales Zhenjiang Hu, National Institute of Informatics * Program Chair: Olivier Danvy, Aarhus University * Local Arrangements Chair: Soichiro Hidaka, National Institute of Informatics * Workshop Co-Chairs: Gabriele Keller, University of New South Wales Derek Dreyer, MPI-SWS * Programming Contest Chair: Eijiro Sumii, Tohoku University * Publicity Chair: Wouter Swierstra, Radboud Universiteit Nijmegen ===================================================================== From michal.ptaszek@REDACTED Tue Jul 26 11:43:36 2011 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Tue, 26 Jul 2011 11:43:36 +0200 Subject: [erlang-questions] problem with starting gproc In-Reply-To: <1311618863.26599.YahooMailRC@web25405.mail.ukl.yahoo.com> References: <1311618863.26599.YahooMailRC@web25405.mail.ukl.yahoo.com> Message-ID: <4FD9A903-961E-4701-99E8-01FA6C9EDC20@erlang-solutions.com> Hi! it could be worth starting SASL before gproc - to see what is causing crash of the application. Try running: application:start(sasl). application:start(gproc). Best regards, Michal Ptaszek On Jul 25, 2011, at 8:34 PM, Roman Shestakov wrote: > hello, > > does anybody else have a problem with gproc? > > I started having some issues since I upgraded to Lion, the fresh gproc build from git head wouldn't start. ( I rebuilt erlang install after macos 10.7 but it didn't help) > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.5 (abort with ^G) > 1> application:start(gproc). > > =INFO REPORT==== 25-Jul-2011::19:26:14 === > application: gproc > exited: {shutdown,{gproc_app,start,[normal,[]]}} > type: temporary > {error,{shutdown,{gproc_app,start,[normal,[]]}}} > 2> > any ideas what might be causing it ? > > I wander if it is my machine problem or generic issue. > > Regards, Roman > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freza@REDACTED Tue Jul 26 15:18:50 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 26 Jul 2011 14:18:50 +0100 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: <20110726131850.GA1370@hanele> # Guy Wiener 2011-07-26: > Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? No. > Alt., is there some what to encourage Continuation-Passing Style programming in Erlang, other than > just recommending that all consequent function calls will be in tail-call position? Not sure I understand this bit. What do you want to use CPS for, specifically? Maybe people can then suggest Erlangy ways to achieve that goal. BR, -- Jachym From ponton@REDACTED Tue Jul 26 15:31:45 2011 From: ponton@REDACTED (Tomasz Maciejewski) Date: Tue, 26 Jul 2011 15:31:45 +0200 Subject: [erlang-questions] common_test test naming Message-ID: Hello! Why does common_test name differently tests run with -suite and -spec/-dir? For example: a) "-suite lib/foo/bar_SUITE" is named lib.foo.bar_SUITE with only one suite b) "-dir lib/foo" is named lib.foo and has all the suits in this directory The result is two entries in the HTML log: lib.foo.bar_SUITE and lib.foo (with bar_SUITE) What I would like to do, is to have ability to run entire test spec as well as just a specific test suite. -- Tomasz Maciejewski From kenneth.lundin@REDACTED Tue Jul 26 15:38:28 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 26 Jul 2011 15:38:28 +0200 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: References: Message-ID: I suppose you arleady know that there is a tftp implementation included in the Erlang/OTP distribution. You find the documentation here. http://www.erlang.org/doc/man/tftp.html /Kenneth Erlang/OTP Ericsson On Tue, Jul 26, 2011 at 9:36 AM, Anupam Kapoor wrote: > hi all, > > i have been playing with an implementation of tftp > (rfc-1350). encoding/decoding tftp-pdu's is trivial with erlang's bit > syntax. > > there seem to be couple of approaches to encoding/decoding tftp-pdu's: > > ? 1. use native "in-place" encode/decode > ? 2. have a 'generic' encode/decode routine which accepts > ? ? ? ? - a list of pdu primitives e.g. 'byte'/'string' etc. and > ? ? ? ? - a list of corresponding values > ? ? ?and returns the binary packet. > > ? ? ?for example, encoding a tftp-data packet looks like this: > ? ? ? ? ?,---- > ? ? ? ? ?| %% encode data packet > ? ? ? ? ?| encode_data_pkt(BlkNum, Data) -> > ? ? ? ? ?| ? ? encode_packet([?BYTE, ?BYTE, ?RAW], > [?TFTP_OPCODE_DATA, BlkNum, Data]). > ? ? ? ? ?`---- > > to me, approach #2 seems pretty close to rfc's packet-format > specification. for example, the data-packet is shown as > > ? ? ? ? ? 2 bytes ? ? 2 bytes ? ? ?n bytes > ? ? ? ? ? ---------------------------------- > ? ? ? ? ?| Opcode | ? Block # ?| ? Data ? ? | > ? ? ? ? ? ---------------------------------- > > can you please provide your suggestion on whether #2 might be a good > approach to follow for complicated protocols e.g. gtp-v2 which defines a > large number of messages ? also, are there better (increasing order of > readability, performance etc.) approaches for doing this ? > > thank you > kind regards > anupam > > ps: the attached file contains encoders for tftp messages... > > -- > In the beginning was the lambda, and the lambda was with Emacs, and > Emacs was the lambda. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From erlang@REDACTED Tue Jul 26 16:19:56 2011 From: erlang@REDACTED (erlang) Date: Tue, 26 Jul 2011 16:19:56 +0200 Subject: [erlang-questions] team of programmers for hire Message-ID: <4E2ECD0C.702@silversoft.pl> Hello, Please excuse me for this bit of advertisement. I'll keep it short. I'd like to inform you that our team of 6 Erlang programmers has some free man-power. If you have projects that need expert Erlang programmers, please contact me. Some facts: - 50+ years of cumulative programming experience - 15+ years of cumulative Erlang experience - we work in both fixed cost and hourly fee For more information, drop me an email. Sincerly yours, Ewa Woznicka Executive Assistant SilverSoft Sp. z o.o. http://silversoft.pl/index.php?lng=en From anupam.kapoor@REDACTED Tue Jul 26 17:26:00 2011 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Tue, 26 Jul 2011 20:56:00 +0530 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: References: Message-ID: yes i know. but my question was more towards a better approach towards writing packet encoders/decoders. may you please let me know your thoughts on that ? kind regards anupam On Tue, Jul 26, 2011 at 7:08 PM, Kenneth Lundin wrote: > I suppose you arleady know that there is a tftp implementation included in the > Erlang/OTP distribution. > > You find the documentation here. > http://www.erlang.org/doc/man/tftp.html > > /Kenneth Erlang/OTP Ericsson > > On Tue, Jul 26, 2011 at 9:36 AM, Anupam Kapoor wrote: >> hi all, >> >> i have been playing with an implementation of tftp >> (rfc-1350). encoding/decoding tftp-pdu's is trivial with erlang's bit >> syntax. >> >> there seem to be couple of approaches to encoding/decoding tftp-pdu's: >> >> ? 1. use native "in-place" encode/decode >> ? 2. have a 'generic' encode/decode routine which accepts >> ? ? ? ? - a list of pdu primitives e.g. 'byte'/'string' etc. and >> ? ? ? ? - a list of corresponding values >> ? ? ?and returns the binary packet. >> >> ? ? ?for example, encoding a tftp-data packet looks like this: >> ? ? ? ? ?,---- >> ? ? ? ? ?| %% encode data packet >> ? ? ? ? ?| encode_data_pkt(BlkNum, Data) -> >> ? ? ? ? ?| ? ? encode_packet([?BYTE, ?BYTE, ?RAW], >> [?TFTP_OPCODE_DATA, BlkNum, Data]). >> ? ? ? ? ?`---- >> >> to me, approach #2 seems pretty close to rfc's packet-format >> specification. for example, the data-packet is shown as >> >> ? ? ? ? ? 2 bytes ? ? 2 bytes ? ? ?n bytes >> ? ? ? ? ? ---------------------------------- >> ? ? ? ? ?| Opcode | ? Block # ?| ? Data ? ? | >> ? ? ? ? ? ---------------------------------- >> >> can you please provide your suggestion on whether #2 might be a good >> approach to follow for complicated protocols e.g. gtp-v2 which defines a >> large number of messages ? also, are there better (increasing order of >> readability, performance etc.) approaches for doing this ? >> >> thank you >> kind regards >> anupam >> >> ps: the attached file contains encoders for tftp messages... >> >> -- >> In the beginning was the lambda, and the lambda was with Emacs, and >> Emacs was the lambda. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. From joelr1@REDACTED Tue Jul 26 17:40:16 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Jul 2011 16:40:16 +0100 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: References: Message-ID: <2AA8603D-66D0-4C3E-81C0-02DD1A05EBC3@gmail.com> Maybe this? http://lambda-the-ultimate.org/node/2243 and my Erlang implementation http://wagerlabs.com/parsing-text-and-binary-files-with-erlang On Jul 26, 2011, at 4:26 PM, Anupam Kapoor wrote: > yes i know. but my question was more towards a better approach towards > writing packet encoders/decoders. may you please let me know your > thoughts on that ? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From freza@REDACTED Tue Jul 26 20:18:49 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 26 Jul 2011 19:18:49 +0100 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: References: Message-ID: <20110726181849.GA21579@hanele> # Anupam Kapoor 2011-07-26: > yes i know. but my question was more towards a better approach towards > writing packet encoders/decoders. may you please let me know your > thoughts on that ? It's hard to answer such a broad question more precisely, so just some random considerations really. The usual general approach is that you design native Erlang representation of the PDUs involved and then provide functions with about this signature: decode(binary()) -> PDU encode(PDU) -> iolist() Popular choices of native representations ('PDU' above) include[*]: * TVLs, that is [{Key, Val}, ...] where Key tends to be atom. * Plain tuples if PDUs are simple enough. * Records, usually one record type per PDU type. Or various combinations of these (ie. use record to represent anything in packet header and then have 'body' element which is a TVL holding decoded packet body). And sometimes it makes sense to separate packet header processing from payload processing altogether (pass payload around as a binary and decode it later on -- or never). The API of your protocol stack would usually operate on the native 'PDU' type, so that the user doesn't have to encode/decode things him/herself, but this again depends. HTH, -- Jachym [*] I'm intentionally not listing dict in there although I've seen it used a few times. It looks horribly ugly in logs, the performance gain is questionable and often irrelevant, it's hard to iterate over, impossible to pattern-match on... From joelr1@REDACTED Tue Jul 26 21:07:19 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Jul 2011 20:07:19 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto Message-ID: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Did I miss a lively and heated discussion? http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html Bring it on! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From bob@REDACTED Tue Jul 26 21:20:41 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 26 Jul 2011 12:20:41 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: "Funargs: Ruby-like blocks for Erlang" is the discussion thread that resulted in that blog post: http://erlang.org/pipermail/erlang-questions/2011-July/060104.html On Tue, Jul 26, 2011 at 12:07 PM, Joel Reymont wrote: > Did I miss a lively and heated discussion? > > http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html > > Bring it on! > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joelr1@REDACTED Tue Jul 26 21:22:51 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Jul 2011 20:22:51 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: On Jul 26, 2011, at 8:20 PM, Bob Ippolito wrote: > "Funargs: Ruby-like blocks for Erlang" is the discussion thread that > resulted in that blog post: Yes, I've been dutifully ignoring it for the past few days :-). This post warrants a new discussion then! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Tue Jul 26 21:45:08 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Jul 2011 20:45:08 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: I'll offer a datapoint? I hate Erlang, although it's been a great way for me to make money. OpenPoker, for example, keeps on giving and is being licensed by a major game development company. To Tony's list of Erlang warts I would also add the complete lack of transparency during performance optimization. This has bit me during OpenPoker development and also during the "Hot wheels" optimization contest [1]. You do have the facility for call counting (eprof) as well as for per-process profiling with fprof. A regular Erlang system these days will likely have thousands of processes running, processing requests and returning a response. What you get with fprof is a useless profiling listing of thousands of processes. What you really want is to know what happens along the request processing path. It took N seconds to process the request. Where did this time go? You want to average this over hundreds of requests and see where the bottlenecks are. Do you really want this? You are out of luck, I'm sorry. No tools exist to help you. That said, I recently came off a project where we build an ad serving platform with OCaml and ZeroMQ. I pushed for this technology stack as I wanted to see if this is a suitable replacement for Erlang. The short answer is no. We ended up defining tons of Google Protocol Buffers messages for everything we needed and writing boring serialization and deserialization code. Also, what would normally be an Erlang process turned out to be a separate executable. There was some loss of compilation speed as the number of these executables multiplied. Still, it was quite manageable until we got to sharing data. We started by using plain Redis. This required a server to "front" Redis and process various requests, updating other processes of state changes via ZeroMQ. Serializing things to strings and back is an incredible pain in the ass! Later, it turned out that Redis was not the ideal choice since a lot of the long-running processes required most of the data in Redis and it was too slow to suck it out upon startup of each of these processes. We ended up dumping data into "bootstrap files", easily and quickly loadable into OCaml. What I'm trying to say here is that our application clearly benefited from code running "on top" of the data, e.g. Erlang processes using ETS or Mnesia. Our scores of Protobuf messages would have become simple Erlang tuples or records. Data sharing would have become a non-issue. I wish Tony good luck with his search for the Erlang replacement. I haven't found one yet. [1] https://groups.google.com/d/topic/erlang-programming/2pRrWneJwG8/overview -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From fritchie@REDACTED Tue Jul 26 22:00:19 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 26 Jul 2011 15:00:19 -0500 Subject: [erlang-questions] How to differentiate info_msg and error_msg In-Reply-To: Message of "Tue\, 26 Jul 2011 14\:13\:12 +0800." Message-ID: <91151.1311710419@snookles.snookles.com> Andy W. Song wrote: as> I can print messages to error log using error_logger:info_msg and? as> error_logger:error_msg. But I can't find a way to enable only as> error_msg while mask info_msg, just like log level in a lot of Linux as> software.? Andy, that feature isn't available in the SASL default error handler. To have that kind of control, you need to write your own event handler or use someone else's. I'm a bit partial to Lager (released recently by Basho), but at least one or two others were mentioned in this recent discussiont on this list: https://groups.google.com/group/erlang-programming/browse_thread/thread/817da017b5af2e1b -Scott From fred.hebert@REDACTED Tue Jul 26 22:05:03 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 26 Jul 2011 16:05:03 -0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: <70B1A8AD-929F-4042-84AC-944BF6181795@erlang-solutions.com> Yes, you did: http://erlang.org/pipermail/erlang-questions/2011-July/thread.html#60104 -- Fred H?bert http://www.erlang-solutions.com On 2011-07-26, at 15:07 PM, Joel Reymont wrote: > Did I miss a lively and heated discussion? > > http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html > > Bring it on! > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fred.hebert@REDACTED Tue Jul 26 22:06:05 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 26 Jul 2011 16:06:05 -0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <70B1A8AD-929F-4042-84AC-944BF6181795@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <70B1A8AD-929F-4042-84AC-944BF6181795@erlang-solutions.com> Message-ID: <4AFCA9B1-D734-475E-A547-5FD61667DC1C@erlang-solutions.com> Woops, sorry for the very late reply. It appears the e-mails got clogged in the mail server and I thought it was brand new. Disregard that e-mail and the previous one. -- Fred H?bert http://www.erlang-solutions.com On 2011-07-26, at 16:05 PM, Fr?d?ric Trottier-H?bert wrote: > Yes, you did: http://erlang.org/pipermail/erlang-questions/2011-July/thread.html#60104 > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-07-26, at 15:07 PM, Joel Reymont wrote: > >> Did I miss a lively and heated discussion? >> >> http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html >> >> Bring it on! >> >> -------------------------------------------------------------------------- >> - for hire: mac osx device driver ninja, kernel extensions and usb drivers >> ---------------------+------------+--------------------------------------- >> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont >> ---------------------+------------+--------------------------------------- >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Tue Jul 26 22:23:54 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 26 Jul 2011 22:23:54 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: On Tue, Jul 26, 2011 at 21:45, Joel Reymont wrote: > What you really want is to know what happens along the request processing path. It took N seconds to process the request. Where did this time go? You want to average this over hundreds of requests and see where the bottlenecks are. Do you really want this? You are out of luck, I'm sorry. No tools exist to help you. I agree this thing would be really neat to have. To make it somewhat efficient, I'd definitely propose an sFlow-like approach. Add a flag to erlang:trace/3 such that we can trace a function based upon a sampling value. For instance, that 1/8192 messages on average is traced. Together with call/return we now know how much time was spent in that function as a whole. If you also add dependent trace probes (if the 1/8192 trigger, then these should be enabled as well) you should in principle be able to build such a tool out of the trace facilities. It is *almost* there. -- J. From joelr1@REDACTED Tue Jul 26 22:51:06 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 26 Jul 2011 21:51:06 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: On Jul 26, 2011, at 9:23 PM, Jesper Louis Andersen wrote: > Add a flag to erlang:trace/3 such that we can trace a function based upon a > sampling value. For instance, that 1/8192 messages on average is > traced. Together with call/return we now know how much time was spent > in that function as a whole. There may be message passing along the request path, processes talking to one another. You want to capture and time this interaction. You can do all this by capturing and processing gigabytes of trace messages, figuring out where the request starts and ends and taking it from there. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From essen@REDACTED Tue Jul 26 23:05:05 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 26 Jul 2011 23:05:05 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: <4E2F2C01.3060801@dev-extend.eu> On 07/26/2011 09:07 PM, Joel Reymont wrote: > Did I miss a lively and heated discussion? > > http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html > > Bring it on! All I'm seeing there is "Make Erlang like language XYZ". Not that Erlang can't learn from other languages and platforms, I'm sure it does, but Erlang focuses on reliability and you can't be reliable and add new trendy features every year while remaining reliable. Adding features to Erlang is a long and progressive process and this sort of development pace doesn't with the "following trends" approach most programmers use with regards to programming languages. Two days ago Java was trendy, yesterday it was Ruby, today it's Javascript, tomorrow it'll be another, everytime reinventing the loop or partially taking from other languages. Personally I like the slow but certain approach better. So yeah, maybe Erlang doesn't have the best features of Lisp, and Clojure, and Java, and Ruby, and Javascript, and-- but neither do them. I'm a firm believer of the "right tool for the right job", and it turns out that Erlang actually is good for most server application jobs, so it pleases me quite well. And it doesn't need to have 10 million features to fit that role, it just needs to be reliable, concurrent and distributed. I disagree about Erlang's syntax though. Not that it doesn't suck, but I don't know any language syntax which isn't sucking (I'm sure you'll tell me your favorite language has great syntax). They all have issues, they all have traps you must learn to avoid, and nothing you can do will change that. Maybe you can add some syntax sugar to avoid typing 5 characters here and there, but seriously what's the point? If the syntax changes don't make me think differently about the way I program, and don't allow me to do things I couldn't do before, then they're pretty much irrelevant to the matter at hand: getting things done. There *are* interesting concepts that could be good, one day, in Erlang. But not having them doesn't prevent us from being efficient so they should go after making Erlang even more reliable, and also after adding missing library components to the pool of applications. Unless you're doing your project for fun or something, in which case have fun. But toying with syntax doesn't make much sense if you need to bring in money. Erlang does very well what it's designed for, and I'm thankful for that. I'm using Erlang to do a lot of things. I wouldn't use Erlang to do everything. But Erlang allows me to get the work done and more importantly it allows me to move on to other projects quickly because Erlang provides me with everything needed to avoid almost all possible bugs (Dialyzer, PropEr, eunit and ct are wonderful to work with), and once in production I'm confident my applications will always be running and available. That's something I can't say for any other language or platform out there today. -- Lo?c Hoguin Dev:Extend From jesper.louis.andersen@REDACTED Wed Jul 27 00:14:42 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 27 Jul 2011 00:14:42 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: On Tue, Jul 26, 2011 at 22:51, Joel Reymont wrote: > > On Jul 26, 2011, at 9:23 PM, Jesper Louis Andersen wrote: > >> Add a flag to erlang:trace/3 such that we can trace a function based upon a >> sampling value. For instance, that 1/8192 messages on average is >> traced. Together with call/return we now know how much time was spent >> in that function as a whole. > > There may be message passing along the request path, processes talking to one another. > > You want to capture and time this interaction. Ah yes, that is something I completely missed. Indeed you want to trace along messages also, but that makes it less obvious what to do. I am more for capturing such instrumentation information on a random sampling basis. You could do it manually though by inserting instrumentation functions along the path and then use those functions as hooks for a unique tag. That is probably what I'd do today if I needed this. Then I'd trace the instrumentation functions at random intervals. -- J. From watson.timothy@REDACTED Wed Jul 27 00:53:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 26 Jul 2011 23:53:13 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: > To Tony's list of Erlang warts I would also add the complete lack of transparency during performance optimization. This has bit me during OpenPoker development and also during the "Hot wheels" optimization contest [1]. > > You do have the facility for call counting (eprof) as well as for per-process profiling with fprof. A regular Erlang system these days will likely have thousands of processes running, processing requests and returning a response. What you get with fprof is a useless profiling listing of thousands of processes. > > What you really want is to know what happens along the request processing path. It took N seconds to process the request. Where did this time go? You want to average this over hundreds of requests and see where the bottlenecks are. Do you really want this? You are out of luck, I'm sorry. No tools exist to help you. > Better profiling tool support would be nice. A few people started trying to pull together something that would provide better visibility - my own project, nodewatch, which is heavily based on eper and more recently Richard Jones and friends produced https://github.com/beamspirit/bigwig during the spawnfest competition. I've stopped work on nodewatch for now, to see whether bigwig will move faster - I don't have that much spare time and if others are doing the same thing as their day job then I needn't bother. My ultimate aim was to provide configurable aggregated stats as well as trace support with time budgeting in mind (e.g., how long in the web tier, how long hitting the back end, etc). From ok@REDACTED Wed Jul 27 04:50:37 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 27 Jul 2011 14:50:37 +1200 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: <2AA8603D-66D0-4C3E-81C0-02DD1A05EBC3@gmail.com> References: <2AA8603D-66D0-4C3E-81C0-02DD1A05EBC3@gmail.com> Message-ID: There is an intermediate position between - manually writing packing and unpacking code - representing the type structure as a data structure and interpreting it at run time and that is + writing a "pickler compiler" in Erlang that takes a data structure representing a packet structure and generates Erlang source code for packing and unpacking, which you can then call. From anupam.kapoor@REDACTED Wed Jul 27 05:12:08 2011 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Wed, 27 Jul 2011 08:42:08 +0530 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: <2AA8603D-66D0-4C3E-81C0-02DD1A05EBC3@gmail.com> References: <2AA8603D-66D0-4C3E-81C0-02DD1A05EBC3@gmail.com> Message-ID: On Tue, Jul 26, 2011 at 9:10 PM, Joel Reymont wrote: ,---- | > Maybe this? | > | > http://lambda-the-ultimate.org/node/2243 | > | > and my Erlang implementation | > | > http://wagerlabs.com/parsing-text-and-binary-files-with-erlang `---- thank you ! this is very very cool. with the tftp thingie, i have tried to come up with something similar, though not as 'neat'. the paper reference is also *very* useful. kind regards anupam -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. From anupam.kapoor@REDACTED Wed Jul 27 06:15:59 2011 From: anupam.kapoor@REDACTED (Anupam Kapoor) Date: Wed, 27 Jul 2011 09:45:59 +0530 Subject: [erlang-questions] Fwd: encoding packets a style+efficiency question In-Reply-To: References: <528E26C0-DA5D-4ED2-B20C-53D4D44A1740@cs.otago.ac.nz> Message-ID: sorry, hit a reply, instead of a 'reply-all'. kind regards anupam ---------- Forwarded message ---------- From: Anupam Kapoor Date: Wed, Jul 27, 2011 at 8:54 AM Subject: Re: [erlang-questions] encoding packets a style+efficiency question To: Richard O'Keefe On Wed, Jul 27, 2011 at 8:08 AM, Richard O'Keefe wrote: ,---- | Why did you write | | encode_pkt_string(EncodedPkt, [FirstFmt | RestFmt], [FirstVal | RestVal]) when FirstFmt =:= ?BYTE -> | | instead of | | encode_pkt_string(EncodedPkt, [?BYTE | RestFmt], [FirstVal | RestVal]) -> | | (and similarly with ?STRING and ?RAW)? `---- oooh nice. i didn't know i could do that ! ignorance is not a good defence, is it ? ,---- | Come to that, why use ?BYTE ?STRING and ?RAW instead of just plain | byte, string, and raw? `---- well, to confess, i was initially thinking of having a bunch of custom-format specifiers e.g. ? ?"%B" would represent a byte ? ?"%S" would represent a string etc. the user would then specify the "format" of their respective packets, and get those stamped out via actual encoders for the primitive types. things would go in the reverse direction during decode operations. unlike C (vsnprintf and friends), i don't know *how* to do that in erlang, which is when i wrote what you see in the attached file. having done that, i realized that there is no need to have 'strings' and changed those to atoms. kind regards anupam -- In the beginning was the lambda, and the lambda was with Emacs, and Emacs was the lambda. From jwatte@REDACTED Wed Jul 27 06:23:41 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 26 Jul 2011 21:23:41 -0700 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: One of the biggest problems of CPS code in Twisted or Node or most other frameworks using that mechanism, is that it ends up with single-threaded servers, making poor use of multi-threaded hardware. In Erlang, a process is comparable to the cost as a continuation with a function lambda in other languages (give or take a little bit, depending on specifics of implementation and use). Write your code serially, in a process, and talk to it using messaging. It's easier to write, debug, and understand that way. Because of data being immutable, there is no risk of threading data hazards. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 26, 2011 at 12:39 AM, Guy Wiener wrote: > Hello everyone, > Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? > Alt., is there some what to encourage Continuation-Passing Style > programming in Erlang, other than just recommending that all consequent > function calls will be in tail-call position? > > Thanks, > Guy > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 27 06:30:06 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 26 Jul 2011 21:30:06 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> Message-ID: FWIW: We tag each application-level request with a "what" and "when" and pass this record along to other parts of the system. When the application gets back its reply, it computes statistics. This allows us to track messages that take surprisingly long time -- typically based on some external dependency -- and alert on how many they are, and get some coarse-level statistics. The most fun is when the request enters through one node, but finishes/exits through another node, because the clocks are not 100% in sync, so we sometimes end up with messages taking negative time. The fix for that is to also tag the "source node" in the per-request record that we carry along, and finish off by forwarding that record back to the creator for measurement. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 26, 2011 at 3:14 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Jul 26, 2011 at 22:51, Joel Reymont wrote: > > > > On Jul 26, 2011, at 9:23 PM, Jesper Louis Andersen wrote: > > > >> Add a flag to erlang:trace/3 such that we can trace a function based > upon a > >> sampling value. For instance, that 1/8192 messages on average is > >> traced. Together with call/return we now know how much time was spent > >> in that function as a whole. > > > > There may be message passing along the request path, processes talking to > one another. > > > > You want to capture and time this interaction. > > Ah yes, that is something I completely missed. Indeed you want to > trace along messages also, but that makes it less obvious what to do. > I am more for capturing such instrumentation information on a random > sampling basis. You could do it manually though by inserting > instrumentation functions along the path and then use those functions > as hooks for a unique tag. That is probably what I'd do today if I > needed this. Then I'd trace the instrumentation functions at random > intervals. > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed Jul 27 06:34:04 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 26 Jul 2011 21:34:04 -0700 Subject: [erlang-questions] encoding packets a style+efficiency question In-Reply-To: References: Message-ID: Your "encode_packet()" really isn't that different from io_lib:format(), except perhaps when you want to generate binary representations. io_lib:format() takes a format string and arguments, and returns the formatted data, much like vsprintf() in C. Personally, I prefer to code the binaries myself, probably shallowly wrapped in a module of encode and decode functions (much like the PDU suggestion above). For the one case where we rolled our own protocol, we ended up choosing Google protocol buffers, and wrote an erlang encoder/decoder code generator for protoc. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Jul 26, 2011 at 12:36 AM, Anupam Kapoor wrote: > hi all, > > i have been playing with an implementation of tftp > (rfc-1350). encoding/decoding tftp-pdu's is trivial with erlang's bit > syntax. > > there seem to be couple of approaches to encoding/decoding tftp-pdu's: > > 1. use native "in-place" encode/decode > 2. have a 'generic' encode/decode routine which accepts > - a list of pdu primitives e.g. 'byte'/'string' etc. and > - a list of corresponding values > and returns the binary packet. > > for example, encoding a tftp-data packet looks like this: > ,---- > | %% encode data packet > | encode_data_pkt(BlkNum, Data) -> > | encode_packet([?BYTE, ?BYTE, ?RAW], > [?TFTP_OPCODE_DATA, BlkNum, Data]). > `---- > > to me, approach #2 seems pretty close to rfc's packet-format > specification. for example, the data-packet is shown as > > 2 bytes 2 bytes n bytes > ---------------------------------- > | Opcode | Block # | Data | > ---------------------------------- > > can you please provide your suggestion on whether #2 might be a good > approach to follow for complicated protocols e.g. gtp-v2 which defines a > large number of messages ? also, are there better (increasing order of > readability, performance etc.) approaches for doing this ? > > thank you > kind regards > anupam > > ps: the attached file contains encoders for tftp messages... > > -- > In the beginning was the lambda, and the lambda was with Emacs, and > Emacs was the lambda. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Jul 27 06:42:19 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 27 Jul 2011 16:42:19 +1200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E2F2C01.3060801@dev-extend.eu> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> Message-ID: <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> On 27/07/2011, at 9:05 AM, Lo?c Hoguin wrote: > On 07/26/2011 09:07 PM, Joel Reymont wrote: >> Did I miss a lively and heated discussion? >> >> http://www.unlimitednovelty.com/2011/07/trouble-with-erlang-or-erlang-is-ghetto.html >> >> Bring it on! There are some substantive issues in that blog entry. (1) Frames have not been implemented yet. Now if I had provided a model implementation, things might have been different. Or if Joe had provided a model implementation of his earlier and fundamentally similar "proper structs", again things might have been different. My excuse is that the BEAM architecture is simply not documented anywhere that I can find, and I have too many other things to do to grovel around in the guts in Erlang to figure it out. So let me add my item here: (2) BEAM has no usable documentation. One reason this is important is because some people have liked the ideas underneath Erlang well enough to try to build other syntaxes for it. But that is inexcusably hard with BEAM undocumented. (3) There doesn't seem to be anything in the Erlang _approach_ that should interfere with scaling, but the _implementation_ appears not to scale well much past 16 cores. I don't know if that is current information. If true, it's an important limitation of the implementation which I'm sure will be given a lot of attention. I don't expect it to stay true. (4) He doesn't seem to like the Erlang garbage collector, but that's something which has changed more than once, and he does not offer any actual _measurements_. I tried the experiment of allocating 1,000,000,000 list cells (but never keeping more than 10,000 of them at a time). Erlang, byte-codes: 7.58 seconds (= 7.58 nsec/allocation). Erlang, native : 3.92 seconds (= 3.92 nsec/allocation). Java -O -server : 11.40 seconds (= 11.40 nsec/allocation). Java -O -client : 12.26 seconds (= 12.26 nsec/allocation). Java has come a long way. I don't have an Azul system to try. He praised tcmalloc. I note that it only recently became usable without pain on my laptop (MacOS X) and that building it produced reams of warning messages about using a deprecated interface, so it may not work much longer. It doesn't work at all on the other machine on my desk. (Erlang works on both.) I wrote a similar benchmark in C and linked it with libtcmalloc.a. I killed that program after it had run for more than 10 times as long as the Erlang code. So when he says "Erlang ... can't take advantage of libraries like tcmalloc", there doesn't appear to be any advantage that Erlang *could* take. In short, Erlang's memory management is criticised, and it MAY be that this is justified, but the blog entry provides no EVIDENCE. By the way, savour the irony. "Erlang's approach [of] using separate heaps per process", which he criticises, is in fact used elsewhere: ptmalloc does it, the tcmalloc documentation makes it absolutely clear that tcmalloc does this also (more precisely, it uses a per-thread cache, which is what the "tc" part of the name means), and some recent Java systems have done the same thing, with a per-thread cache for memory management, backed by a shared heap. The point of the per- thread cache is to reduce locking. There is a spectrum of approaches from nothing shared to everything shared, and it seems clear that everyone sees merit in not being at either extreme. Progress must be driven by measurement. (5) He doesn't like HiPE. For myself, I don't _care_ whether HiPE is a JIT or a jackal, as long as it gives me a useful improvement in performance. Inlining across module boundaries _has_ been tried, I believe (there's a paper about it somewhere), but it's hard to reconcile with hot loading. HiPE was, of course, a project contributed by "the community", and depended on funding which I believe has come to an end. Anyone who wants a better compiler should try to find funds for it. Sun and Google have vastly deeper pockets than Kostis Sagonas! I think it is particularly unfair to criticise HiPE for having a limited range of back ends when it works on *MORE* systems than the tcmalloc library he praised. (6) Erlang is not general purpose. But it was never intended to be, and isn't advertised as such. In fact Erlang loves state, but it wants state to be encapsulated within processes. "What should you do if you want to deal with a shared-state concurrency program in Erlang?" Lie down until the feeling passes off. If you want shared-state concurrency, I can tell you where to find Ada. I can tell you where to find concurrent ML (Mlton does it OK). I can tell you where to find Haskell, which is actually pretty amazing these days. One can respect Erlang without being married to it. (7) He doesn't like the syntax. Well, it's not quite as much of a disaster as Java syntax, and for sure it's not as ugly as CAML or F#. (They are so ugly that they make SML look beautiful, and for someone who prefers Haskell to SML on aesthetic grounds, that's saying a lot.) The funny thing is that what makes Erlang syntax clunky is precisely its *similarity* to classical languages like Pascal and C... Given documentation for BEAM, we might get more alternative syntaxes to play with. It's fair to point out that Erlang resulted from an experiment with several approaches, so responsible steps were taken to make sure that it wasn't _too_ bad. (8) He criticises immutable state on the grounds that while you can share tails of a list, you can't share prefixes or infixes. The answer, of course, is multifold: - there are immutable data structures where you *can* share infixes and you *can* use them in Erlang, it's just that they don't have built in syntax. (For that matter, it would be possible to implement Erlang so that slices of tuples could be shared just like slices of strings in Java. SML does this. In fact, Concurrent SML would answer so many of his issues that I'm surprised he didn't mention it.) - this is only a problem if you *want* to share prefixes or infixes, and somehow I never do - in languages with mutable state you cannot safely share ANYTHING. The argument that allocating pure objects is a bad match for modern hardware is a non sequitur. Let me quote a paper about Fork/Join parallelism for Java: "In many ways, modern GC facilities are perfect matches to fork/join frameworks: These programs can generate enormous numbers of tasks, nearly all of which quickly turn into garbage after they are executed." That is, generating enormous amounts of garbage can be a >good< thing, provided it's the kind that garbage collectors manage well. I've been on the garbage collection mailing list for a while, and the Memory Management proceedings have contained papers showing that garbage collection can be *worse* for locality (and thus modern hardware) and papers showing that it can be *better* for locality (and thus modern hardware). What this means is that one cannot simply *assume* "side effects = good for cache, immutability = bad", one must *measure*. (9) He doesn't like the standard library. Interestingly enough, I hear the same kind of thing in the Haskell mailing list about the Haskell "standard Prelude". And there is a project to develop a new standard Prelude. I believe there is general agreement that an improved Erlang library could be developed and would be very nice to have. (I've always found the differences between say ETS and DETS more confusing than helpful.) This is something that can be done piecemeal and by individuals. The things I would like to say about the Java libraries would have to be displayed on asbestos screens... Heck, I like the Ada libraries less than I used to; the additions are *pointful* but not to my mind *tasteful*. And so it goes. There *are* things about Erlang that can be improved. Some things *have* been improved, some things are being improved, and some things don't need anyone to wait for Ericsson to do them. From mjtruog@REDACTED Wed Jul 27 08:08:15 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 26 Jul 2011 23:08:15 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> Message-ID: <4E2FAB4F.7000903@gmail.com> On 07/26/2011 09:42 PM, Richard O'Keefe wrote: > (3) There doesn't seem to be anything in the Erlang _approach_ that > should interfere with scaling, but the _implementation_ appears > not to scale well much past 16 cores. > > I don't know if that is current information. If true, it's an > important limitation of the implementation which I'm sure will be > given a lot of attention. I don't expect it to stay true. Any problems scaling on systems with more than 16 cores just seems to be related to the current cost of those systems (would love to know what the tests currently show on the Tileras > 16 cores though). Erlang appears to have much more natural scalability when you compare it to Java, and the criticism of the Erlang garbage collector is unsubstantiated (as previously mentioned). You can not say an approach is wrong because it isn't the Java-way, or perhaps the Sun-way. The per-process garbage collection avoids central state, so it encourages scalability with a design for parallelism. Throwing a ton of money at Azul to push a single heap garbage collector beyond normal limits might sound fun, but it only shows how long and drawn out technical failure can be (like Sun's stock price was for instance, an almost perfect bell curve). > (4) He doesn't seem to like the Erlang garbage collector, but that's > something which has changed more than once, and he does not > offer any actual _measurements_. > > I tried the experiment of allocating 1,000,000,000 list cells > (but never keeping more than 10,000 of them at a time). > Erlang, byte-codes: 7.58 seconds (= 7.58 nsec/allocation). > Erlang, native : 3.92 seconds (= 3.92 nsec/allocation). > Java -O -server : 11.40 seconds (= 11.40 nsec/allocation). > Java -O -client : 12.26 seconds (= 12.26 nsec/allocation). > > Java has come a long way. I don't have an Azul system to try. > > He praised tcmalloc. I note that it only recently became usable > without pain on my laptop (MacOS X) and that building it produced > reams of warning messages about using a deprecated interface, so > it may not work much longer. It doesn't work at all on the other > machine on my desk. (Erlang works on both.) I wrote a similar > benchmark in C and linked it with libtcmalloc.a. I killed that > program after it had run for more than 10 times as long as the > Erlang code. So when he says "Erlang ... can't take advantage of > libraries like tcmalloc", there doesn't appear to be any > advantage that Erlang *could* take. > > In short, Erlang's memory management is criticised, and it MAY be > that this is justified, but the blog entry provides no EVIDENCE. > > By the way, savour the irony. "Erlang's approach [of] using > separate heaps per process", which he criticises, is in fact > used elsewhere: ptmalloc does it, the tcmalloc documentation > makes it absolutely clear that tcmalloc does this also (more > precisely, it uses a per-thread cache, which is what the "tc" > part of the name means), and some recent Java systems have > done the same thing, with a per-thread cache for memory > management, backed by a shared heap. The point of the per- > thread cache is to reduce locking. There is a spectrum of > approaches from nothing shared to everything shared, and it > seems clear that everyone sees merit in not being at either > extreme. Progress must be driven by measurement. It is hard to believe that there is controversy still here. Per-process garbage collection avoids shared state which avoids the need for low-level locking which makes Erlang scalable. To argue for some other approach in Erlang seems like idiocy to me, because it wouldn't be a real Actor model that can provide fault-tolerance (actually keep failures isolated). Why would you want to fool around with some broken fake Actor implementation that is unscalable (like your operating system, for instance)? Seems like a waste of time, just like this garbage collector complaint. Yes, HiPE has issues, Erlang is not meant for all programs, and the syntax is different. Nothing is perfect. Not even the standard library is perfect. I have yet to hear of a perfect standard library in any language. I think having problems with a standard library is a natural problem because the people that write the standard library impose a taxonomy on functionality that not everyone shares naturally, because we are not psychic. Learning the standard library just seems like a natural process in computer science when you get to relate a standard library to the others you had the misfortune to use in the past. Why bother complaining about a taxonomy that is just as bad as any other? If it serves the purpose it was designed for well, then there is no reason to care, it just happens to be different from what you might expect based on your own limited knowledge. > And so it goes. > > There *are* things about Erlang that can be improved. > Some things *have* been improved, some things are being improved, > and some things don't need anyone to wait for Ericsson to do them. The frustration with these issues seems natural and common, both on this mailing list and elsewhere. However, I think it is important to be proactive rather than giving in to emotional arguments that lack justification or evidence. From ulf.wiger@REDACTED Wed Jul 27 09:16:20 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 09:16:20 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E2FAB4F.7000903@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> Message-ID: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> On 27 Jul 2011, at 08:08, Michael Truog wrote: > On 07/26/2011 09:42 PM, Richard O'Keefe wrote: >> (3) There doesn't seem to be anything in the Erlang _approach_ that >> should interfere with scaling, but the _implementation_ appears >> not to scale well much past 16 cores. >> >> I don't know if that is current information. If true, it's an >> important limitation of the implementation which I'm sure will be >> given a lot of attention. I don't expect it to stay true. > > Any problems scaling on systems with more than 16 cores just seems to be related to the current cost of those systems (would love to know what the tests currently show on the Tileras > 16 cores though). I guess it's fair to say that the Erlang/OTP doesn't have a "dangerously sexy" approach to SMP scalability. They recognise that their sponsors [1] would have their heads if BEAM started hanging or dumping core in the sole interest of scaling to ridiculously many cores on hardware that none of the sponsors are using. Instead, they try to deliver incremental improvements without ever sacrificing the rock-solid performance of BEAM that Erlang users have come to expect. They're not batting 100% in that regard, but close enough to be very impressive, IMHO. They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) What is interesting for most Erlang users (I think), is not how well Erlang can scale with synthetic benchmarks, but on *real applications*, where lots of other factors affect the result, and scalability is only one parameter (albeit a very important one). Having had the privilege of participating in key roles throughout the development lifecycle of a number of telecom-class Erlang-based systems, I've come to think that constant-factor nuisances (like syntax quibbles) are simply unimportant. What matters in these projects is that you get the fundamentals right, esp. in regard to messaging, fault-tolerance and provisioning. If you don't, you will never be able to control schedules and costs, and your product is unlikely to ever be profitable. In this environment, very few programming languages are even considered: usually, it's either C/C++ or Java, and in some companies, Erlang as well. The .NET languages are usually ruled out because their commercial viability is largely tied to the Windows platform, and such a single-platform focus is an issue when you expect your product to serve for decades. Ruby, Python etc. can be useful for writing test scripts, and perhaps some supporting tools, or perhaps even some peripheral component in the system, but (at this point in time) not much more than that. I realise that most people today are not in this environment, but rather developing small [2] projects, often in areas where Ruby, Python, Clojure, Scala, Haskell et al are perfectly viable alternatives. I think it's great that the Erlang/OTP team is getting feedback from this crowd, and even pressure to compete with the "coolest" languages out there. Erlang will be better for it. But let's do this with some mutual respect. I'm not even going to begin commenting on the "Erlang cargo culters" references, and find it curious that Erlang is so often said to "suck", even though the same people often admit that there is no viable replacement - isn't this much more of a criticism against those other languages, which apparently don't just suck enough to give discomfort, but to the point of being near unusable in this realm? Having been in the position, some years ago, where I was thinking about changing jobs, and couldn't really find any jobs where I could make good use of my Erlang experience (even if not in Erlang), I find today's software market much more exciting - where other languages are drawing inspiration from Erlang, and in some cases really challenging it, even in the concurrency domain. [1] Sponsors = commercial projects within Ericsson that use Erlang to build 5-nines-style complex messaging products for commercial gain. They typically need to use NEBS-compliant ATCA hardware from reputable vendors, which is standardised across the corporation to minimise supply chain costs. In these settings, I would expect most systems to be running on 4- or 8-core machines today, perhaps starting to move up to 16-core in a few places. [2] To me, anything under 100 KLOC is small. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From max.lapshin@REDACTED Wed Jul 27 09:34:13 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 27 Jul 2011 11:34:13 +0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> Message-ID: On Wed, Jul 27, 2011 at 11:16 AM, Ulf Wiger wrote: > > I guess it's fair to say that the Erlang/OTP doesn't have a "dangerously sexy" approach to SMP scalability. They recognise that their sponsors [1] would have their heads if BEAM started hanging or dumping core in the sole interest of scaling to ridiculously many cores on hardware that none of the sponsors are using. Instead, they try to deliver incremental improvements without ever sacrificing the rock-solid performance of BEAM that Erlang users have come to expect. They're not batting 100% in that regard, but close enough to be very impressive, IMHO. > Many people forget about it. Erlang messaging system seems to be 99,999% bug free. Scala is full of silly bugs. I'd preferer to build software, that I sell on bug free platform and great thanks to your team for it. The only thing that really affects me is lack of profiling tools or lack of documentation on them. From thomasl_erlang@REDACTED Wed Jul 27 11:07:44 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 27 Jul 2011 02:07:44 -0700 (PDT) Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> Message-ID: <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> ----- Original Message ----- > From: Ulf Wiger > To: Michael Truog > Cc: "erlang-questions@REDACTED Questions" > Sent: Wednesday, July 27, 2011 9:16 AM > Subject: Re: [erlang-questions] trouble with erlang or erlang is a ghetto > > ... > I guess it's fair to say that the Erlang/OTP doesn't have a > "dangerously sexy" approach to SMP scalability. They recognise that > their sponsors [1] would have their heads if BEAM started hanging or dumping > core in the sole interest of scaling to ridiculously many cores on hardware that > none of the sponsors are using. Furthermore, IMO it's something of a mistake to obsess over single-process SMP. I think distributed erlang is pretty sexy.? Best, Thomas From wiener.guy@REDACTED Wed Jul 27 12:05:57 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Wed, 27 Jul 2011 13:05:57 +0300 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: In a normal scenario, I would prefer serial code, admittedly. However, the scenario that I want to implement is the following: I want to retry the same piece of code several times, each time with a different settings. The ways to do so are either making sure that each step in the code is a function (i.e., CPS), or exploiting callcc. Other ideas? G. On Wed, Jul 27, 2011 at 7:23 AM, Jon Watte wrote: > One of the biggest problems of CPS code in Twisted or Node or most other > frameworks using that mechanism, is that it ends up with single-threaded > servers, making poor use of multi-threaded hardware. > In Erlang, a process is comparable to the cost as a continuation with a > function lambda in other languages (give or take a little bit, depending on > specifics of implementation and use). > Write your code serially, in a process, and talk to it using messaging. > It's easier to write, debug, and understand that way. Because of data being > immutable, there is no risk of threading data hazards. > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Tue, Jul 26, 2011 at 12:39 AM, Guy Wiener wrote: > >> Hello everyone, >> Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? >> Alt., is there some what to encourage Continuation-Passing Style >> programming in Erlang, other than just recommending that all consequent >> function calls will be in tail-call position? >> >> Thanks, >> Guy >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Jul 27 12:18:28 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 27 Jul 2011 12:18:28 +0200 Subject: [erlang-questions] [erlang-bugs] Code should not compile In-Reply-To: References: Message-ID: 2011/7/27 Gordon Guthrie : > I attach a module and a header file. > > The module hmac_api_lib.erl has a bug on line 9 - the define is > incomplete. It should not compile, but it does. Which erlang version are you using? As far as I remember, it was not necessary to close macros with ")." up to R13B, but the R14B at least warns about it (maybe doesn't even compile - I'm not sure). From hvjunk@REDACTED Wed Jul 27 12:21:05 2011 From: hvjunk@REDACTED (Hendrik Visage) Date: Wed, 27 Jul 2011 12:21:05 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> Message-ID: On Wed, Jul 27, 2011 at 11:07 AM, Thomas Lindgren wrote: > ----- Original Message ----- >> From: Ulf Wiger > >> ... >> I guess it's fair to say that the Erlang/OTP doesn't have a >> "dangerously sexy" approach to SMP scalability. They recognise that >> their sponsors [1] would have their heads if BEAM started hanging or dumping >> core in the sole interest of scaling to ridiculously many cores on hardware that >> none of the sponsors are using. > > > Furthermore, IMO it's something of a mistake to obsess over single-process SMP. I think distributed erlang is pretty sexy. And *that* my dear friends, is where the "clouds" are hanging out ;) A rack in each city centre with a bunch of 1u/2u boxes, using something like GoogleFS, and we have close to a petabytes of distributed disaster tolerant storage and processing, and then I also say: Bring me my Erlang :) From dmitrii@REDACTED Wed Jul 27 12:27:10 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 27 Jul 2011 13:27:10 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> Message-ID: <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> On Jul 27, 2011, at 10:16 AM, Ulf Wiger wrote: > > On 27 Jul 2011, at 08:08, Michael Truog wrote: > >> On 07/26/2011 09:42 PM, Richard O'Keefe wrote: >>> (3) There doesn't seem to be anything in the Erlang _approach_ that >>> should interfere with scaling, but the _implementation_ appears >>> not to scale well much past 16 cores. >>> >>> I don't know if that is current information. If true, it's an >>> important limitation of the implementation which I'm sure will be >>> given a lot of attention. I don't expect it to stay true. >> >> Any problems scaling on systems with more than 16 cores just seems to be related to the current cost of those systems (would love to know what the tests currently show on the Tileras > 16 cores though). > > > I guess it's fair to say that the Erlang/OTP doesn't have a "dangerously sexy" approach to SMP scalability. They recognise that their sponsors [1] would have their heads if BEAM started hanging or dumping core in the sole interest of scaling to ridiculously many cores on hardware that none of the sponsors are using. Instead, they try to deliver incremental improvements without ever sacrificing the rock-solid performance of BEAM that Erlang users have come to expect. They're not batting 100% in that regard, but close enough to be very impressive, IMHO. > > They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) > Also don't forget that there's a grant given to universities to further develop Erlang to become massively multicore, as Kostis Sagonas said at Erlang Factory in London =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Wed Jul 27 12:39:17 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 27 Jul 2011 13:39:17 +0300 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: Something along the lines of next(_, []) -> ok; next(Fun, [H|T]) -> F = fun() Fun(T) end, Result = Fun(H), {Result, F}. test() -> Params = [[1], [1, 2], [1, 2, 3]], Start = next(fun erlang:length/1, Params), output(Start). output(ok) -> io:format("ok"); output({Result, F}) -> io:format("~w", [Result]), NewResult = F(), output(NewResult). ? > In a normal scenario, I would prefer serial code, admittedly. > However, the scenario that I want to implement is the following: I want to retry the same piece of code several times, each time with a different settings. The ways to do so are either making sure that each step in the code is a function (i.e., CPS), or exploiting callcc. > Other ideas? > > G. > > On Wed, Jul 27, 2011 at 7:23 AM, Jon Watte wrote: > One of the biggest problems of CPS code in Twisted or Node or most other frameworks using that mechanism, is that it ends up with single-threaded servers, making poor use of multi-threaded hardware. > In Erlang, a process is comparable to the cost as a continuation with a function lambda in other languages (give or take a little bit, depending on specifics of implementation and use). > Write your code serially, in a process, and talk to it using messaging. It's easier to write, debug, and understand that way. Because of data being immutable, there is no risk of threading data hazards. > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > On Tue, Jul 26, 2011 at 12:39 AM, Guy Wiener wrote: > Hello everyone, > Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? > Alt., is there some what to encourage Continuation-Passing Style programming in Erlang, other than just recommending that all consequent function calls will be in tail-call position? > > Thanks, > Guy > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Wed Jul 27 13:29:18 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 13:29:18 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> Message-ID: On 27 Jul 2011, at 12:27, Dmitrii Dimandt wrote: >> They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) >> > > > Also don't forget that there's a grant given to universities to further develop Erlang to become massively multicore, as Kostis Sagonas said at Erlang Factory in London Well, that's what I was referring to, but there will be a more coordinated announcement soon. :) ?That, and another one, with a different slant. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From lukas.larsson@REDACTED Wed Jul 27 14:06:00 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Wed, 27 Jul 2011 13:06:00 +0100 (BST) Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E2FAB4F.7000903@gmail.com> Message-ID: Here[1] is a pretty resent publication about testing Erlang on a TILEPro64. It also contains a quite pretty detailed description of how the beam VM works internally with it's scheduling and memory allocations algorithms. Lukas [1] http://kth.diva-portal.org/smash/record.jsf?pid=diva2:392243 ----- Original Message ----- From: "Michael Truog" To: "erlang-questions@REDACTED Questions" Sent: Wednesday, 27 July, 2011 8:08:15 AM Subject: Re: [erlang-questions] trouble with erlang or erlang is a ghetto On 07/26/2011 09:42 PM, Richard O'Keefe wrote: > (3) There doesn't seem to be anything in the Erlang _approach_ that > should interfere with scaling, but the _implementation_ appears > not to scale well much past 16 cores. > > I don't know if that is current information. If true, it's an > important limitation of the implementation which I'm sure will be > given a lot of attention. I don't expect it to stay true. Any problems scaling on systems with more than 16 cores just seems to be related to the current cost of those systems (would love to know what the tests currently show on the Tileras > 16 cores though). Erlang appears to have much more natural scalability when you compare it to Java, and the criticism of the Erlang garbage collector is unsubstantiated (as previously mentioned). You can not say an approach is wrong because it isn't the Java-way, or perhaps the Sun-way. The per-process garbage collection avoids central state, so it encourages scalability with a design for parallelism. Throwing a ton of money at Azul to push a single heap garbage collector beyond normal limits might sound fun, but it only shows how long and drawn out technical failure can be (like Sun's stock price was for instance, an almost perfect bell curve). > (4) He doesn't seem to like the Erlang garbage collector, but that's > something which has changed more than once, and he does not > offer any actual _measurements_. > > I tried the experiment of allocating 1,000,000,000 list cells > (but never keeping more than 10,000 of them at a time). > Erlang, byte-codes: 7.58 seconds (= 7.58 nsec/allocation). > Erlang, native : 3.92 seconds (= 3.92 nsec/allocation). > Java -O -server : 11.40 seconds (= 11.40 nsec/allocation). > Java -O -client : 12.26 seconds (= 12.26 nsec/allocation). > > Java has come a long way. I don't have an Azul system to try. > > He praised tcmalloc. I note that it only recently became usable > without pain on my laptop (MacOS X) and that building it produced > reams of warning messages about using a deprecated interface, so > it may not work much longer. It doesn't work at all on the other > machine on my desk. (Erlang works on both.) I wrote a similar > benchmark in C and linked it with libtcmalloc.a. I killed that > program after it had run for more than 10 times as long as the > Erlang code. So when he says "Erlang ... can't take advantage of > libraries like tcmalloc", there doesn't appear to be any > advantage that Erlang *could* take. > > In short, Erlang's memory management is criticised, and it MAY be > that this is justified, but the blog entry provides no EVIDENCE. > > By the way, savour the irony. "Erlang's approach [of] using > separate heaps per process", which he criticises, is in fact > used elsewhere: ptmalloc does it, the tcmalloc documentation > makes it absolutely clear that tcmalloc does this also (more > precisely, it uses a per-thread cache, which is what the "tc" > part of the name means), and some recent Java systems have > done the same thing, with a per-thread cache for memory > management, backed by a shared heap. The point of the per- > thread cache is to reduce locking. There is a spectrum of > approaches from nothing shared to everything shared, and it > seems clear that everyone sees merit in not being at either > extreme. Progress must be driven by measurement. It is hard to believe that there is controversy still here. Per-process garbage collection avoids shared state which avoids the need for low-level locking which makes Erlang scalable. To argue for some other approach in Erlang seems like idiocy to me, because it wouldn't be a real Actor model that can provide fault-tolerance (actually keep failures isolated). Why would you want to fool around with some broken fake Actor implementation that is unscalable (like your operating system, for instance)? Seems like a waste of time, just like this garbage collector complaint. Yes, HiPE has issues, Erlang is not meant for all programs, and the syntax is different. Nothing is perfect. Not even the standard library is perfect. I have yet to hear of a perfect standard library in any language. I think having problems with a standard library is a natural problem because the people that write the standard library impose a taxonomy on functionality that not everyone shares naturally, because we are not psychic. Learning the standard library just seems like a natural process in computer science when you get to relate a standard library to the others you had the misfortune to use in the past. Why bother complaining about a taxonomy that is just as bad as any other? If it serves the purpose it was designed for well, then there is no reason to care, it just happens to be different from what you might expect based on your own limited knowledge. > And so it goes. > > There *are* things about Erlang that can be improved. > Some things *have* been improved, some things are being improved, > and some things don't need anyone to wait for Ericsson to do them. The frustration with these issues seems natural and common, both on this mailing list and elsewhere. However, I think it is important to be proactive rather than giving in to emotional arguments that lack justification or evidence. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From valentin@REDACTED Wed Jul 27 14:24:07 2011 From: valentin@REDACTED (Valentin Micic) Date: Wed, 27 Jul 2011 14:24:07 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: <065284BE-5B90-44DC-BAC3-CE405B2FCCF5@pixie.co.za> On 27 Jul 2011, at 2:06 PM, Lukas Larsson wrote: > Yes, HiPE has issues, Erlang is not meant for all programs, and the syntax is different. Nothing is perfect. Particularly programers... V/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Wed Jul 27 14:58:42 2011 From: mihai@REDACTED (Mihai Balea) Date: Wed, 27 Jul 2011 08:58:42 -0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> Message-ID: <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> On Jul 27, 2011, at 7:29 AM, Ulf Wiger wrote: > > On 27 Jul 2011, at 12:27, Dmitrii Dimandt wrote: > >>> They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) >>> >> >> >> Also don't forget that there's a grant given to universities to further develop Erlang to become massively multicore, as Kostis Sagonas said at Erlang Factory in London > > Well, that's what I was referring to, but there will be a more coordinated announcement soon. :) > > > ?That, and another one, with a different slant. Are you referring to this announcement? http://www.erlang-solutions.com/press-releases/3/entry/1253 Mihai From dmitrii@REDACTED Wed Jul 27 15:11:25 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 27 Jul 2011 16:11:25 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> Message-ID: <1C4D8656-4825-4377-91FF-7897DE90D5D8@dmitriid.com> On Jul 27, 2011, at 3:58 PM, Mihai Balea wrote: > > On Jul 27, 2011, at 7:29 AM, Ulf Wiger wrote: > >> >> On 27 Jul 2011, at 12:27, Dmitrii Dimandt wrote: >> >>>> They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) >>>> >>> >>> >>> Also don't forget that there's a grant given to universities to further develop Erlang to become massively multicore, as Kostis Sagonas said at Erlang Factory in London >> >> Well, that's what I was referring to, but there will be a more coordinated announcement soon. :) >> >> >> ?That, and another one, with a different slant. > > Are you referring to this announcement? > > http://www.erlang-solutions.com/press-releases/3/entry/1253 Oh wow. Congrats to Erlang Solutions! From ulf.wiger@REDACTED Wed Jul 27 15:38:39 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 15:38:39 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> Message-ID: <1A5D1384-751B-4DF6-BF61-E68131514A6B@erlang-solutions.com> Yeah, something like that. Being on vacation, I haven't kept up with the announcements. :) BR, Ulf W On 27 Jul 2011, at 14:58, Mihai Balea wrote: > > On Jul 27, 2011, at 7:29 AM, Ulf Wiger wrote: > >> >> On 27 Jul 2011, at 12:27, Dmitrii Dimandt wrote: >> >>>> They *have* run benchmarks on 64-core machines, but mainly to learn about what's around the corner, and understand what changes will be needed to get there. You will soon hear about some pretty exciting developments in the area of Erlang for massive scalability, but there is a time and a place for everything? ;-) >>>> >>> >>> >>> Also don't forget that there's a grant given to universities to further develop Erlang to become massively multicore, as Kostis Sagonas said at Erlang Factory in London >> >> Well, that's what I was referring to, but there will be a more coordinated announcement soon. :) >> >> >> ?That, and another one, with a different slant. > > Are you referring to this announcement? > > http://www.erlang-solutions.com/press-releases/3/entry/1253 > > Mihai Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From alisdairsullivan@REDACTED Wed Jul 27 16:06:28 2011 From: alisdairsullivan@REDACTED (Alisdair Sullivan) Date: Wed, 27 Jul 2011 07:06:28 -0700 Subject: [erlang-questions] Unicode noncharacter inconsistency Message-ID: The binary Unicode 'switches' /utf8, /utf16, etc raise an error when asked to encode u+fffe or u+ffff (which are non characters, making this a defensible and perhaps sensible position), but not when asked to encode u+1fffe, u+1ffff, u+2fffe, ..., u+10ffff. I considered submitting a bug, but as these are ok for internal implementations and are just forbidden during interchange, it's arguable this does adhere to the spec. Is there some rationale behind this inconsistency, or is it simply an oversight? The reserved non characters u+fdd0 - u+fdef should also probably behave as u+fffe/ffff, as they are in the same category. From ulf.wiger@REDACTED Wed Jul 27 16:27:17 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 16:27:17 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <1C4D8656-4825-4377-91FF-7897DE90D5D8@dmitriid.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <6D346E74-8827-4481-A222-3766F63F1BBF@dmitriid.com> <1B7D6874-6A15-4732-BCFC-8378089D99C2@hates.ms> <1C4D8656-4825-4377-91FF-7897DE90D5D8@dmitriid.com> Message-ID: <156CAE17-24FC-4939-85FE-1570ACA69033@erlang-solutions.com> On 27 Jul 2011, at 15:11, Dmitrii Dimandt wrote: >> Are you referring to this announcement? >> >> http://www.erlang-solutions.com/press-releases/3/entry/1253 > > > Oh wow. Congrats to Erlang Solutions! Congrats to Erlang in general! This is a major infusion of research money on two fronts: low-level multicore performance, and high-level SMP and large-cluster performance. Ericsson is a key player in the RELEASE project, as is Kostis, with one leg in Uppsala and one in Athens. University of Kent continues its erlang-related research and Heriot-Watt University picks it up again after a hiatus. ;-) Another important commercial player is EDF, adding both expertise and case studies on massively scaled discrete-time simulations. One ESL angle will be using Moebius to enable capability-based orchestration and testing of large heterogeneous clusters. The two projects also have great collaboration potential, partly due to the common Erlang ingredient, but also due to good ties between Heriot-Watt, Kent and S:t Andrews. Here is a factsheet for the RELEASE project: http://www.macs.hw.ac.uk/~trinder/RELEASEfactsheet.pdf No corresponding factsheed for Paraphrase, I'm afraid. More detailed announcements will follow. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Wed Jul 27 18:54:03 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 27 Jul 2011 09:54:03 -0700 Subject: [erlang-questions] Los Angeles Erlang User Group Meetup 8/2/2011 7PM Message-ID: <20110727165403.GA15285@alumni.caltech.edu> Hi, Just wanted to get the word out about the next LA Erlang Meetup, next Tuesday. The meetup will be in Pasadena at OpenX (where we use quite a bit of erlang). Jay Nelson from Duomark will be presenting. Information and RSVP can be found here http://www.erlang.la/events/21550021/?eventId=21550021&action=detail Hope to see some of you there, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From joelr1@REDACTED Wed Jul 27 21:00:09 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Jul 2011 20:00:09 +0100 Subject: [erlang-questions] tracing function calls in a given process Message-ID: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> What is the right dbg incarnation for tracing functions calls within a given process? There's dbg:p(Pid, [all]). to trace all messages in a process. There's dbg:tpl(M, F, dbg:fun2ms(fun(_) -> return_trace() end)), to trace all calls to a function in a given module, or dbg:tpl(M, dbg:fun2ms(fun(_) -> return_trace() end)), to trace all function calls in a module. How do I combine the above, though? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From attila.r.nohl@REDACTED Wed Jul 27 21:11:08 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 27 Jul 2011 21:11:08 +0200 Subject: [erlang-questions] tracing function calls in a given process In-Reply-To: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> References: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> Message-ID: 2011/7/27 Joel Reymont : > What is the right dbg incarnation for tracing functions calls within a given process? > > There's > > ? ?dbg:p(Pid, [all]). > > to trace all messages in a process. There's > > ? ?dbg:tpl(M, F, dbg:fun2ms(fun(_) -> return_trace() end)), > > to trace all calls to a function in a given module, or > > ? ?dbg:tpl(M, dbg:fun2ms(fun(_) -> return_trace() end)), > > to trace all function calls in a module. > > How do I combine the above, though? I don't know, but if you don't find a solution, you can always write your own trace handler - the Pid, Module and Function is there in the trace messages, so it is possible to filter on them. From vladdu55@REDACTED Wed Jul 27 21:16:20 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 27 Jul 2011 21:16:20 +0200 Subject: [erlang-questions] tracing function calls in a given process In-Reply-To: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> References: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> Message-ID: Hi, On Wed, Jul 27, 2011 at 21:00, Joel Reymont wrote: > What is the right dbg incarnation for tracing functions calls within a given process? > > There's > ? ?dbg:p(Pid, [all]). > to trace all messages in a process. There's > ? ?dbg:tpl(M, F, dbg:fun2ms(fun(_) -> return_trace() end)), > to trace all calls to a function in a given module, or > ? ?dbg:tpl(M, dbg:fun2ms(fun(_) -> return_trace() end)), > to trace all function calls in a module. > How do I combine the above, though? ? ?dbg:p(Pid, [all]). will enable all tracing, messages, function calls, the works. Use ? ?dbg:p(Pid, [c]). to just trace function calls. http://www.erlang.org/doc/man/dbg.html#p-2 will give you all the details. regards, Vlad From joelr1@REDACTED Wed Jul 27 21:24:00 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Jul 2011 20:24:00 +0100 Subject: [erlang-questions] tracing function calls in a given process In-Reply-To: References: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> Message-ID: <3514CBBB-2F22-4C9D-B38C-73939B5A8C83@gmail.com> On Jul 27, 2011, at 8:16 PM, Vlad Dumitrescu wrote: > dbg:p(Pid, [all]). > will enable all tracing, messages, function calls, the works. This did not trace calls for me, hence my question. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From vladdu55@REDACTED Wed Jul 27 21:33:41 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 27 Jul 2011 21:33:41 +0200 Subject: [erlang-questions] tracing function calls in a given process In-Reply-To: <3514CBBB-2F22-4C9D-B38C-73939B5A8C83@gmail.com> References: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> <3514CBBB-2F22-4C9D-B38C-73939B5A8C83@gmail.com> Message-ID: Weird, it's what I use most of the time... Does [c] work? /Vlad On Wed, Jul 27, 2011 at 21:24, Joel Reymont wrote: > > On Jul 27, 2011, at 8:16 PM, Vlad Dumitrescu wrote: > >> ? ?dbg:p(Pid, [all]). >> will enable all tracing, messages, function calls, the works. > > This did not trace calls for me, hence my question. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > From joelr1@REDACTED Wed Jul 27 21:35:14 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Jul 2011 20:35:14 +0100 Subject: [erlang-questions] tracing function calls in a given process In-Reply-To: References: <72B3A682-1DF8-404C-B220-7A042C7A09ED@gmail.com> Message-ID: On Jul 27, 2011, at 8:16 PM, Vlad Dumitrescu wrote: > dbg:p(Pid, [all]). > will enable all tracing, messages, function calls, the works. I just realized what I'm doing wrong. I need to combine dog:p with dog:tp, then it works. The keywords are "according to the trace patterns". --- c (call) Traces global function calls for the process according to the trace patterns set in the system (see tp/2). --- -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Jul 27 21:45:52 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Jul 2011 20:45:52 +0100 Subject: [erlang-questions] ip trace port performance Message-ID: Is the ip trace port speedy enough to collect all trace messages in a running system? I'm talking all function calls and messages. Will this bring the system under test to its knees or just result in a significant slowdown? How much of a slow down am I looking at? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ulf.wiger@REDACTED Wed Jul 27 22:08:58 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 22:08:58 +0200 Subject: [erlang-questions] ip trace port performance In-Reply-To: References: Message-ID: <5892CBCF-345B-4DE2-895C-6CE63ED2CF9E@erlang-solutions.com> The trace port will drop trace messages rather than hurt performance. I don't know the exact algorithm, but most likely, it imposes a queue limit on the outgoing port, or something. BR, Ulf W On 27 Jul 2011, at 21:45, Joel Reymont wrote: > Is the ip trace port speedy enough to collect all trace messages in a running system? > > I'm talking all function calls and messages. > > Will this bring the system under test to its knees or just result in a significant slowdown? > > How much of a slow down am I looking at? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Wed Jul 27 22:23:17 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 22:23:17 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> Message-ID: <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> On 27 Jul 2011, at 18:58, Richard Bucker wrote: > There is nothing in the language or VM that makes it's sigma rating any better or worse than any other language or framework. Erlang doesn't pretend to have a sigma rating. What language does (except perhaps in some very narrow context)? Erlang has been used to build systems with better than 5-nines availability, according the rules for measuring In-Service Performance (ISP) in Telecoms. That's what it was made for. > Indeed Mnesia and Mnesia's replication cannot add to it's rating as it fails all the time. And hello-world is not an acceptable app to test or rate. As it happens the system responsible for the 99.99999% availability figure floating around [1], did in fact use mnesia. Klarna uses mnesia, and has best-in-class availability among online factoring services in Scandinavia (I believe - I have no hard data, but their CTO was pretty confident at the last Erlang Factory in London). There are other examples, and just about every database used in anger has its share of horror stories too. Perhaps you shouldn't believe everything you read in the blogosphere? ;-) [1] That was a real data point, made official by British Telecom - which is why it could be used by Joe in a talk at MIT. The average ISP reported from all our deployments was lower, but consistently better than 99.999% while I was keeping track. I cannot give you the exact numbers. > Erlang solves some interesting problems and creates some new ones. It's more of a "religion of functionalism" than anything else. And for the same reasons it's a little dangerous and risky. I'm not sure what you base that on. Erlang's history has clearly been driven by pragmatism, and has made it this far mainly because it proved excellent for solving the problems it was designed for. But I admit - Erlang has gone through some different phases: * from being young and hyped, * to being banned, bashed and nearly killed, * to seeing some kind of truce, where it was tolerated, as long as its proponents didn't try to make a big deal out of it, * to wooing part of the Open Source crowd and seeing some hype again (and some bashing too, for good measure). Having experienced all of those, I have long since given up on trying to convince anyone who doesn't want to be convinced. For the purpose of the list, I will try to comment on things that I find incorrect, and share my experience when asked. If you don't see anything special about Erlang, and consider using it a risk, I'd assume the sensible thing is not to use it. Are you willing to entertain the possibility that other people reach a very different conclusion, without being blinded by religious fervour? BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From max.lapshin@REDACTED Wed Jul 27 22:36:19 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 28 Jul 2011 00:36:19 +0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: Ulf. There maybe many speculations about "most convenient language" or "best garbage collection", but one serious problem is really mentioned. It is lack of data structure, which can survive upgrade of only one part of software. I can't create redistributable plugins for erlyvideo that can use headers with record description, I have to write such things: ems_media:get(Media, last_dts) and frankly speaking I don't understand how do other people live without this. I understand, that C language has the same problem, but maybe there are some ideas to make some delayed instantiation of record usage in modules? I.e. add "record_get(Record, name)" instruction to beam, which should be translated to "element(Record, N)" on loading? From joelr1@REDACTED Wed Jul 27 22:39:04 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 27 Jul 2011 21:39:04 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: <4E760909-560E-46C7-B83A-A22031D9A668@gmail.com> On Jul 27, 2011, at 9:36 PM, Max Lapshin wrote: > I can't create redistributable plugins for erlyvideo that can use > headers with record description, I have to write such things: > ems_media:get(Media, last_dts) > and frankly speaking I don't understand how do other people live without this. What exactly is the problem with this? Records are just tuples, no? P.S. I don't think one needs to post to the Google group as it mirrors this list. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From max.lapshin@REDACTED Wed Jul 27 22:49:01 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 28 Jul 2011 00:49:01 +0400 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E760909-560E-46C7-B83A-A22031D9A668@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> <4E760909-560E-46C7-B83A-A22031D9A668@gmail.com> Message-ID: On Thu, Jul 28, 2011 at 12:39 AM, Joel Reymont wrote: > > What exactly is the problem with this? > > Records are just tuples, no? > Problem happens, when plugin is compiled against version 2, where record rtmp_session was looking like: #rtmp_session{pid, user_id} but launched against version 3, where rtmp_session is: #rtmp_session{pid, user_id, session_id} Yes, records are only tuples and their field names are lost in runtime. From essen@REDACTED Wed Jul 27 22:49:48 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 27 Jul 2011 22:49:48 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E760909-560E-46C7-B83A-A22031D9A668@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> <4E760909-560E-46C7-B83A-A22031D9A668@gmail.com> Message-ID: <4E3079EC.7090302@dev-extend.eu> On 07/27/2011 10:39 PM, Joel Reymont wrote: > > On Jul 27, 2011, at 9:36 PM, Max Lapshin wrote: > >> I can't create redistributable plugins for erlyvideo that can use >> headers with record description, I have to write such things: >> ems_media:get(Media, last_dts) >> and frankly speaking I don't understand how do other people live without this. > > > What exactly is the problem with this? > > Records are just tuples, no? I think that's what confuses most people. They see records as something more, when they're just tuples and aren't designed to be anything else. Records need to be upgraded the same as tuples when changes are made and of course it takes a little more care than struct/frames would. -- Lo?c Hoguin Dev:Extend From ulf.wiger@REDACTED Wed Jul 27 23:10:13 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 27 Jul 2011 23:10:13 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: <92487E25-9B90-4F5F-B2AA-9D8092A1432F@erlang-solutions.com> Max, Yes, records are awkward for sharing across interfaces for this reason. When I wrote the Diameter application (which is now part of OTP), I tried to mitigate the problem in two ways: - Allow the user to pass a proplist as an alternative to the record. This requires a conversion step, and some slight overhead. - Use 'exprecs' (part of http://github.com/esl/parse_trans, although the Diameter app has its own version), to generate accessor functions similar to what you describe. The notation is perhaps not the most beautiful, but you get used to it. Exprecs actually also has some upgrade support, where you can specify different versions of the same record: http://erlang.2086793.n4.nabble.com/RFC-exprecs-and-record-versions-td2114178.html I don't know if anyone has ever used it; I've never received comments on it, as far as I recall. BR, Ulf W On 27 Jul 2011, at 22:36, Max Lapshin wrote: > Ulf. There maybe many speculations about "most convenient language" or > "best garbage collection", but one serious problem is really > mentioned. > > It is lack of data structure, which can survive upgrade of only one > part of software. > > I can't create redistributable plugins for erlyvideo that can use > headers with record description, I have to write such things: > ems_media:get(Media, last_dts) > and frankly speaking I don't understand how do other people live without this. > > I understand, that C language has the same problem, but maybe there > are some ideas to make some delayed instantiation of record usage in > modules? > I.e. add "record_get(Record, name)" instruction to beam, which should > be translated to "element(Record, N)" on loading? Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jul 28 02:01:28 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 28 Jul 2011 12:01:28 +1200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E2FAB4F.7000903@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> Message-ID: <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> On 27/07/2011, at 6:08 PM, Michael Truog wrote: > > Not even the standard library is perfect. I have yet to hear of a perfect standard library in any language. I think having problems with a standard library is a natural problem because the people that write the standard library impose a taxonomy on functionality that not everyone shares naturally, because we are not psychic. Learning the standard library just seems like a natural process in computer science when you get to relate a standard library to the others you had the misfortune to use in the past. Why bother complaining about a taxonomy that is just as bad as any other? If it serves the purpose it was designed for well, then there is no reason to care, it just happens to be different from what you might expect based on your own limited knowledge. Let me offer a two-fold contrast with Eiffel here. When Betrand Meyer designed Eiffel, he didn't just design a language, he designed a *consistent* naming convention to be applied across *all* library classes. From time to time this got revised, but it did mean that there was a *documented* convention: once you had learned the names for one class, you knew a great deal about what the names in another class meant. I tried to follow a similar policy in the Quintus Prolog library. That's the kind of "inconsistency" people talk about when they criticise the Erlang/OTP libraries: same name for different things, different names for same things. That's the contrast that makes Eiffel look good and Erlang look bad. Now for the other contrast. The Eiffel libraries were proprietary, so other Eiffel implementors had to come up with others. Eventually the N.I.C.E. came up with a common but minimal library. Oy was it minimal! The nearest thing there was to a _useful_ Eiffel library was the GOBO library, and one of the Eiffel implementations (as it happened, the one I used) changed too much for the GOBO maintainer to keep up (or me; I stopped using Eiffel). The *coverage* of a library can be more important than its *consistency*. And to be honest, I think coverage is going to drive Erlang/OTP library development more than consistency is. If someone wants to use SCTP, it is probably more important to them to have a finished SCTP module they can use now than to have a beautiful module they might be able to use next year, maybe. It might be interesting to run some sort of poll to see what currently unsupported areas people care about most, particularly the people who would say "I'd like to use Eiffel but I cannot because it does not have library support for ." From essen@REDACTED Thu Jul 28 02:15:40 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 28 Jul 2011 02:15:40 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: <4E30AA2C.60905@dev-extend.eu> On 07/28/2011 02:01 AM, Richard O'Keefe wrote: > It might be interesting to run some sort of poll to see what currently > unsupported areas people care about most, particularly the people who > would say "I'd like to use Eiffel but I cannot because it does not > have library support for ." Proper Unicode support in Erlang would probably get many votes. -- Lo?c Hoguin Dev:Extend From ok@REDACTED Thu Jul 28 02:39:44 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 28 Jul 2011 12:39:44 +1200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: On 28/07/2011, at 12:06 AM, Lukas Larsson wrote: > Here[1] is a pretty resent publication about testing Erlang on a TILEPro64. It also contains a quite pretty detailed description of how the beam VM works internally with it's scheduling and memory allocations algorithms. > > Lukas > > [1] http://kth.diva-portal.org/smash/record.jsf?pid=diva2:392243 Thanks a lot for the reference. You've got to love this sentence from the abstract: scalability can be improved by reducing lock contention. We all _know_ that fire is hot and water is wet, but sometimes it's nice to see it proved... It looks as though the claim that Erlang runs out of steam at 16 is a furphy, and that there are good ideas about how to make it even better. From bob@REDACTED Thu Jul 28 03:04:03 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 27 Jul 2011 18:04:03 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E30AA2C.60905@dev-extend.eu> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> <4E30AA2C.60905@dev-extend.eu> Message-ID: On Wed, Jul 27, 2011 at 5:15 PM, Lo?c Hoguin wrote: > On 07/28/2011 02:01 AM, Richard O'Keefe wrote: >> It might be interesting to run some sort of poll to see what currently >> unsupported areas people care about most, particularly the people who >> would say "I'd like to use Eiffel but I cannot because it does not >> have library support for ." > > Proper Unicode support in Erlang would probably get many votes. UTF8 support works great these days, what else should you need? ;) -bob From essen@REDACTED Thu Jul 28 03:26:17 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 28 Jul 2011 03:26:17 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> <4E30AA2C.60905@dev-extend.eu> Message-ID: <4E30BAB9.9070104@dev-extend.eu> On 07/28/2011 03:04 AM, Bob Ippolito wrote: > On Wed, Jul 27, 2011 at 5:15 PM, Lo?c Hoguin wrote: >> On 07/28/2011 02:01 AM, Richard O'Keefe wrote: >>> It might be interesting to run some sort of poll to see what currently >>> unsupported areas people care about most, particularly the people who >>> would say "I'd like to use Eiffel but I cannot because it does not >>> have library support for ." >> >> Proper Unicode support in Erlang would probably get many votes. > > UTF8 support works great these days, what else should you need? ;) You can output UTF8 as binary, yes. Maybe as strings too (I'm not really using those so I wouldn't know). But to give an example, can you search inside your UTF8 text for the word "trouv?" including all different variants of the ? character (perhaps even just 'e')? Byte search isn't doing any good here. -- Lo?c Hoguin Dev:Extend From bob@REDACTED Thu Jul 28 03:38:28 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 27 Jul 2011 18:38:28 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4E30BAB9.9070104@dev-extend.eu> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> <4E30AA2C.60905@dev-extend.eu> <4E30BAB9.9070104@dev-extend.eu> Message-ID: On Wed, Jul 27, 2011 at 6:26 PM, Lo?c Hoguin wrote: > On 07/28/2011 03:04 AM, Bob Ippolito wrote: >> On Wed, Jul 27, 2011 at 5:15 PM, Lo?c Hoguin wrote: >>> On 07/28/2011 02:01 AM, Richard O'Keefe wrote: >>>> It might be interesting to run some sort of poll to see what currently >>>> unsupported areas people care about most, particularly the people who >>>> would say "I'd like to use Eiffel but I cannot because it does not >>>> have library support for ." >>> >>> Proper Unicode support in Erlang would probably get many votes. >> >> UTF8 support works great these days, what else should you need? ;) > > You can output UTF8 as binary, yes. Maybe as strings too (I'm not really > using those so I wouldn't know). But to give an example, can you search > inside your UTF8 text for the word "trouv?" including all different > variants of the ? character (perhaps even just 'e')? Byte search isn't > doing any good here. It sounds like you want a unicode normalization library, I don't think this is really a search problem. In Python you'd do this with the unicodedata module. You're right that there is nothing that ships with Erlang for this purpose, at least not that I know of. It seems like this might be easy to solve in a third party library, maybe a binding to ICU. At least one of these probably already exists. -bob From jwatte@REDACTED Thu Jul 28 04:09:58 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 27 Jul 2011 19:09:58 -0700 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: It sounds like what you want is just a way to create a closure, not "CPS" per se. Continuation Passing Style is a specific approach to slice workloads into separate function invocations; one use of which is to implement a poor man's threading system. You could use a fun that returns a function to execute with the values bound (a closure), or parameterized modules. make_function(Arg1, Arg2) -> fun(Whatever) -> Whatever * Arg1 + Arg2 end. make_function_test() -> A = make_function(1, 2), B = make_function(3, 4), 3 = A(1), 4 = A(2), 7 = B(1). 10 = B(2). -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jul 27, 2011 at 3:05 AM, Guy Wiener wrote: > In a normal scenario, I would prefer serial code, admittedly. > However, the scenario that I want to implement is the following: I want to > retry the same piece of code several times, each time with a different > settings. The ways to do so are either making sure that each step in the > code is a function (i.e., CPS), or exploiting callcc. > Other ideas? > > G. > > > On Wed, Jul 27, 2011 at 7:23 AM, Jon Watte wrote: > >> One of the biggest problems of CPS code in Twisted or Node or most other >> frameworks using that mechanism, is that it ends up with single-threaded >> servers, making poor use of multi-threaded hardware. >> In Erlang, a process is comparable to the cost as a continuation with a >> function lambda in other languages (give or take a little bit, depending on >> specifics of implementation and use). >> Write your code serially, in a process, and talk to it using messaging. >> It's easier to write, debug, and understand that way. Because of data being >> immutable, there is no risk of threading data hazards. >> >> Sincerely, >> >> jw >> >> -- >> Americans might object: there is no way we would sacrifice our living >> standards for the benefit of people in the rest of the world. Nevertheless, >> whether we get there willingly or not, we shall soon have lower consumption >> rates, because our present rates are unsustainable. >> >> >> >> On Tue, Jul 26, 2011 at 12:39 AM, Guy Wiener wrote: >> >>> Hello everyone, >>> Is there some equivalent to callcc (as in Scheme or Ruby) in Erlang? >>> Alt., is there some what to encourage Continuation-Passing Style >>> programming in Erlang, other than just recommending that all consequent >>> function calls will be in tail-call position? >>> >>> Thanks, >>> Guy >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jul 28 04:23:17 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 27 Jul 2011 19:23:17 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> Message-ID: A rack in each city? Is the Erlang kernel getting more latency tolerant? Btw, when it comes to "cost effective hardware": With regards to hardware: The price/performance curve right now favors two dies with 6 hyperthreaded cores on each, for a total of 24 hardware threads. Intel will soon be selling (if they aren't already) 10, 12 and higher count cores, and you can buy motherboards and core logic with support for four-way and eight-way dies, but the price/performance isn't quite there yet. When it comes to RAM, you get 32 GB of high-quality, high-performance ECC RAM for about $600. That's almost less that some companies charge for shipping :-/ When it comes to disk space, you can pay $100 for a SATA spindle, or $200 for a SAS spindle, or $400 for a SSD disk. However, that SSD disk will have 20x the transaction throughput of the other spindles, so again, there's really no question that any new box should have an SSD. Even large, database server type boxes generally should have SSDs, except for long-term storage nodes (I'm talking dozens of terabytes and up) where transaction throughput simply doesn't matter. We're also getting to the point where 10 Gbps Ethernet is palatable. It's certainly cheaper to get one 10 Gbps port than 10 1 Gbps ports in switching and network hardware. I imagine that a year from now, you'll start seeing 1 Gbps being pushed down to the "connectivity" part of the segment, and 10 Gbps will be default for any new switches, on-motherboard networks, etc. So, the *current* best-value hardware looks something like: 32 GB of RAM 24 hardware threads (in 2-way NUMA, btw -- does BEAM pay attention to memory affinity?) 240 GB SSD, 1 or 2 (RAID-1 for redundancy) Probably 10 Gbps networking Next year's best-value hardware will probably look something like: 64 GB of RAM 40 hardware threads (still with 2-way NUMA) 240 GB SSD, 1 or 2 (RAID-1 for redundancy) (it will be cheaper than this year, but still the "sweet spot" unless you're building RAID 6 volumes or something) Definitely 10 Gbps networking We're all living in the future! :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Jul 27, 2011 at 3:21 AM, Hendrik Visage wrote: > On Wed, Jul 27, 2011 at 11:07 AM, Thomas Lindgren > wrote: > > > ----- Original Message ----- > >> From: Ulf Wiger > > > >> ... > >> I guess it's fair to say that the Erlang/OTP doesn't have a > >> "dangerously sexy" approach to SMP scalability. They recognise that > >> their sponsors [1] would have their heads if BEAM started hanging or > dumping > >> core in the sole interest of scaling to ridiculously many cores on > hardware that > >> none of the sponsors are using. > > > > > > Furthermore, IMO it's something of a mistake to obsess over > single-process SMP. I think distributed erlang is pretty sexy. > > And *that* my dear friends, is where the "clouds" are hanging out ;) > > A rack in each city centre with a bunch of 1u/2u boxes, using > something like GoogleFS, and we have close to a petabytes of > distributed disaster tolerant storage and processing, and then I also > say: Bring me my Erlang :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Jul 28 04:28:00 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 27 Jul 2011 19:28:00 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: > > It is lack of data structure, which can survive upgrade of only one > part of software. > All modern languages have a simple hash map class that is efficient and built-into the language. JavaScript, Python, Ruby, etc. In Erlang, we have dict, and gb_tree, and proplist, and ets, but none of them are as easy to use, and some of them have performance problems once you scale over a few dozen or a few hundred elements, and others (ets) are way overkill for this particular use case. Or would anyone want to argue that "dict" is actually as good/productive as JavaScript objects or Python dicts? Sincerely, jw -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jul 28 10:39:59 2011 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 28 Jul 2011 10:39:59 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> <4E30AA2C.60905@dev-extend.eu> <4E30BAB9.9070104@dev-extend.eu> Message-ID: <4E31205F.8080901@dev-extend.eu> Hello, >> You can output UTF8 as binary, yes. Maybe as strings too (I'm not really >> using those so I wouldn't know). But to give an example, can you search >> inside your UTF8 text for the word "trouv?" including all different >> variants of the ? character (perhaps even just 'e')? Byte search isn't >> doing any good here. > > It sounds like you want a unicode normalization library, I don't think > this is really a search problem. In Python you'd do this with the > unicodedata module. You're right that there is nothing that ships with > Erlang for this purpose, at least not that I know of. It seems like > this might be easy to solve in a third party library, maybe a binding > to ICU. At least one of these probably already exists. Well yeah. Actually I should have just mentioned something simpler like to_upper that produces quite unexpected effects when done wrong in Unicode. I retract my statement though. Michael Uvarov forwarded me off-list to this library that seems to be just what's needed for any kind of Unicode string manipulation, although I didn't test it: https://github.com/freeakk/ux -- Lo?c Hoguin Dev:Extend From hfventer@REDACTED Thu Jul 28 12:00:20 2011 From: hfventer@REDACTED (Heinrich Venter) Date: Thu, 28 Jul 2011 12:00:20 +0200 Subject: [erlang-questions] OTP process startup vs spawn overhead Message-ID: In an attempt to prove that spinning up OTP processes as workers does not incur that much overhead I have done the following with results that are confusing me a bit. [ timer:tc(fun() -> [ Pid ! ok || {ok, Pid} <- [test_gen:start_link()|| _ <- lists:seq(1,10000)] ] end) || _ <- lists:seq(1,10)]. [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn_link(fun() -> receive _ -> ok end end) || _ <- lists:seq(1,10000)] ] end) || _ <- lists:seq(1,10)]. First start 10,000 gen_server with a very simple handle_info that shuts down the process when it receives anything. Do this 10 times measuring the times for each just to get a bit of smoothing. Then do the same with spawn_link. The idea is that we can compare the times and see how much overhead we have for OTP. The problem is that the spawn_link version takes 4 times LONGER to execute! The issue has to be with the way I do the spawn. Can any one please shed some light on this? -]-[einrich From retnuh@REDACTED Thu Jul 28 12:12:16 2011 From: retnuh@REDACTED (Hunter Kelly) Date: Thu, 28 Jul 2011 11:12:16 +0100 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: If I had to guess, it's because you're creating a closure each time through the loop. Try something like recv() -> receive _ -> ok end. And then in loop: spawn_link(?MODULE, recv, []) By the way, list comprehensions can have multiple generator statements, so you don't have to nest them like that. I think you could do: Fun = fun() -> [ Pid ! ok || _ <- lists:seq(1,10000), Pid <- spawn_link(?MODULE, recv, []) ] end, [ timer:tc(Fun) || _ lists:seq(1,10) ]. to make it a bit more reasonable. This will also remove the cost of creating the closure in the timer statement, too. H On Thu, Jul 28, 2011 at 11:00 AM, Heinrich Venter wrote: > In an attempt to prove that spinning up OTP processes as workers does > not incur that much overhead I have done the following with results > that are confusing me a bit. > > [ timer:tc(fun() -> [ Pid ! ok || {ok, Pid} <- > [test_gen:start_link()|| _ <- lists:seq(1,10000)] ] end) || _ <- > lists:seq(1,10)]. > > [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn_link(fun() -> receive > _ -> ok end end) || _ <- lists:seq(1,10000)] ] end) || _ <- > lists:seq(1,10)]. > > > First start 10,000 gen_server with a very simple handle_info that > shuts down the process when it receives anything. ?Do this 10 times > measuring the times for each just to get a bit of smoothing. > Then do the same with spawn_link. ?The idea is that we can compare the > times and see how much overhead we have for OTP. > > The problem is that the spawn_link version takes 4 times LONGER to > execute! ?The issue has to be with the way I do the spawn. > Can any one please shed some light on this? > > -]-[einrich > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Thu Jul 28 12:33:39 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 12:33:39 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> Message-ID: <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> On 28 Jul 2011, at 04:23, Jon Watte wrote: > A rack in each city? Is the Erlang kernel getting more latency tolerant? I assume you are referring to the occasional "node not responding" issues? As far as I know, the kernel doesn't have issues with latency. The problems with Distributed Erlang are related to a heavy-handed backpressure solution, where processes trying to send to the dist_port are simply suspended if the output queue exceeds a given threshold. When the queue falls under the threshold, all suspended processes are resumed. Since the algorithm doesn't differentiate between processes, this fate can befall the net ticker as well. This *has* been improved a bit in the latest release, and I believe more improvements are forthcoming. Simply increasing the thresholds (currently not configurable) should mostly eliminate the problem. > So, the *current* best-value hardware looks something like: > 32 GB of RAM > 24 hardware threads (in 2-way NUMA, btw -- does BEAM pay attention to memory affinity?) > 240 GB SSD, 1 or 2 (RAID-1 for redundancy) > Probably 10 Gbps networking BEAM is starting to make use of NUMA, for example when allowing you to control the binding of schedulers to cores. See e.g. http://www.erlang.org/doc/man/erl.html (search for NUMA) I believe that the current activities are mainly laying the groundwork for some more powerful optimizations, e.g. delayed deallocation, but R14B actually included quite a few improvements already, esp. in regard to locking. http://www.erlang.org/download/otp_src_R14B.readme Still, micro benchmarks have indicated that memory allocation (not least GC meta-data) locking issues mainly start affecting performance somewhere beyond 40 cores. The question is how much this really affects applications on the "usual" hardware of today? One problem is that it's hard to do detailed profiling on complex real-world applications. The issues limiting scalability might well be wholly unrelated to core VM aspects such as GC, scheduling and message passing. In the first SMP experiments with Ericsson's Telephony Gateway Controller, the big bottleneck was the big lock protecting the ports and linked-in drivers. > Next year's best-value hardware will probably look something like: > 64 GB of RAM > 40 hardware threads (still with 2-way NUMA) > 240 GB SSD, 1 or 2 (RAID-1 for redundancy) (it will be cheaper than this year, but still the "sweet spot" unless you're building RAID 6 volumes or something) > Definitely 10 Gbps networking Yes, but one thing I learned while at Ericsson was that NEBS-compliant ATCA processor boards don't exactly stay on the leading edge of processor capacity. The top-end blade servers today seem to host up to two dual- or quad-core processors. This is not to say that everyone has to evolve at the same pace, but the main funding sources for Erlang/OTP tend to follow this path. Now, Joe has publicly mentioned running an application on a 24-core architecture, for which the optimum setup at the time seemed to be 4 erlang nodes - one for each physical CPU. The problems arise when the application isn't embarrassingly parallel, but requires processes to actually interact with each other, sometimes in fairly complex ways. Also, these many-core architectures are still finding their way in terms of memory access architectures, and each vendor has different ideas. The combination of bottlenecks in the VM, limitations of the hardware architecture, and complex interaction patterns, can easily result in emergent behaviour, which can be quite dramatic. My own take on this is that it's something we will have to learn to live with, and I started developing Jobs - a load control framework (http://github.com/esl/jobs) to allow for "traffic regulation" of erlang-based systems similarly to how one achieves quality of service on TCP-based networks (another messaging system). The key, in my experience, is not usually to go as fast as possible, but to deliver reliable and predictable performance that is good enough for the problem at hand. As many have pointed out through the years, Erlang was never about delivering maximum performance. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jul 28 12:37:15 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 28 Jul 2011 12:37:15 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: On Thu, Jul 28, 2011 at 04:28, Jon Watte wrote: > All modern languages have a simple hash map class that is efficient and > built-into the language. JavaScript, Python,?Ruby, etc. > In Erlang, we have dict, and gb_tree, and proplist, and ets, but none of > them are as easy to use, and some of them have performance problems once you > scale over a few dozen or a few hundred elements, and others (ets) are way > overkill for this particular use case. > > Or would anyone want to argue that "dict" is actually as good/productive as > JavaScript objects or Python dicts? JavaScript, Python, Ruby, etc almost all have ephemeral dict structures. In Erlang, the structure must be persistent, like in Clojure/Haskell and so on. This rules out a number of possible structures. As it stands, "dict" is probably the right replacement, with gb_trees for some situations as they have different speed in those. The "dict" does fairly well, but it is implemented in Erlang and fast as it is, it cannot compete with a C-implementation of a ephemeral structure. Should we want to BIF/NIFify such a structure, I'd recommend either a patricia hash table or a HAMT. In fact, that would be an interesting thing to try. Though I wonder how much I can alter the in-memory structure of the tree from Erlang terms - I'd really like some cache efficient packing there for tree nodes. -- J. From ulf.wiger@REDACTED Thu Jul 28 12:37:36 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 12:37:36 +0200 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: Are you running that expression from the shell? If so, one big difference between the two will be how much is interpreted vs compiled code. Specifically, that the fun in the second test is interpreted, whereas the receive loop in the gen_server is compiled. BR, Ulf W On 28 Jul 2011, at 12:00, Heinrich Venter wrote: > In an attempt to prove that spinning up OTP processes as workers does > not incur that much overhead I have done the following with results > that are confusing me a bit. > > [ timer:tc(fun() -> [ Pid ! ok || {ok, Pid} <- > [test_gen:start_link()|| _ <- lists:seq(1,10000)] ] end) || _ <- > lists:seq(1,10)]. > > [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn_link(fun() -> receive > _ -> ok end end) || _ <- lists:seq(1,10000)] ] end) || _ <- > lists:seq(1,10)]. > > > First start 10,000 gen_server with a very simple handle_info that > shuts down the process when it receives anything. Do this 10 times > measuring the times for each just to get a bit of smoothing. > Then do the same with spawn_link. The idea is that we can compare the > times and see how much overhead we have for OTP. > > The problem is that the spawn_link version takes 4 times LONGER to > execute! The issue has to be with the way I do the spawn. > Can any one please shed some light on this? > > -]-[einrich > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From joelr1@REDACTED Thu Jul 28 12:41:47 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Jul 2011 11:41:47 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> Message-ID: <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> On Jul 28, 2011, at 11:33 AM, Ulf Wiger wrote: > The problems with Distributed Erlang are related to a heavy-handed backpressure solution, where processes trying to send to the dist_port are simply suspended if the output queue exceeds a given threshold. When the queue falls under the threshold, all suspended processes are resumed. Since the algorithm doesn't differentiate between processes, this fate can befall the net ticker as well. I thought my net splits were due to heavy process messaging traffic and the net ticker messages falling behind. That didn't quite explain it but what you said does. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ulf.wiger@REDACTED Thu Jul 28 12:51:13 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 12:51:13 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: Message-ID: <2B3B6588-8D65-4421-A080-435F702C3C73@erlang-solutions.com> On 28 Jul 2011, at 02:39, Richard O'Keefe wrote: > > On 28/07/2011, at 12:06 AM, Lukas Larsson wrote: > ... >> >> [1] http://kth.diva-portal.org/smash/record.jsf?pid=diva2:392243 > > ... > > It looks as though the claim that Erlang runs out of steam at 16 > is a furphy, and that there are good ideas about how to make it > even better. Indeed (although I had to look up the word "furphy"). The picture on page 64, illustrating near-perfect scalability of the Big Bang benchmark on a simulated 128-core SPARC [1] is nice - partly because not everyone has access to the fairly pricey Simics simulations, but also because it illustrates how much of the scalability problem on multicore is about negotiating the divide between processor speed and memory bandwidth [2]. One may also ponder, based on this, how important it is to understand the memory access characteristics - and by extension, the dependency patterns - of the application you want to scale. These are exciting times? :) BR, Ulf W [1] I learned from Simics experts that SPARC is easy to scale this way because it is a very symmetrical architecture. In contrast, many other vendors have specific architectures for their 4-, 8-, 16-core systems, and so on, so just tweaking the number of cores is not very meaningful. [2] For those who haven't read the paper, the simulation assumed zero cost for memory access. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Thu Jul 28 13:00:45 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 13:00:45 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> Message-ID: <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> On 28 Jul 2011, at 12:41, Joel Reymont wrote: > > On Jul 28, 2011, at 11:33 AM, Ulf Wiger wrote: > >> The problems with Distributed Erlang are related to a heavy-handed backpressure solution, where processes trying to send to the dist_port are simply suspended if the output queue exceeds a given threshold. When the queue falls under the threshold, all suspended processes are resumed. Since the algorithm doesn't differentiate between processes, this fate can befall the net ticker as well. > > I thought my net splits were due to heavy process messaging traffic and the net ticker messages falling behind. > > That didn't quite explain it but what you said does. Yeah, we (or mainly, Michal Ptaszek) had reason to dig into this fairly recently, and found that tuning can really make a big difference. Still, the whole area should be revisited for smarter overload handling. A particularly interesting fault situation was when this dynamic ended up suspending the rpc server. It could still receive and process requests (spawning dynamic workers for the processing), but was suspended practically every time it tried to send a reply. Eventually, its message queue used up all memory and killed the node. :) Actually, these changes in R14B01 are relevant: OTP-8901 The runtime system is now less eager to suspend processes sending messages over the distribution. The default value of the distribution buffer busy limit has also been increased from 128 KB to 1 MB. This in order to improve throughput. OTP-8912 The distribution buffer busy limit can now be configured at system startup. For more information see the documentation of the erl +zdbbl command line flag. (Thanks to Scott Lystig Fritchie) and possibly also this: OTP-8916 The inet driver internal buffer stack implementation has been rewritten in order to reduce lock contention. (http://www.erlang.org/download/otp_src_R14B01.readme) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Jul 28 13:03:12 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Jul 2011 12:03:12 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> Message-ID: <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> On Jul 28, 2011, at 12:00 PM, Ulf Wiger wrote: > Yeah, we (or mainly, Michal Ptaszek) had reason to dig into this fairly recently, and found that tuning can really make a big difference. Tuning what? > A particularly interesting fault situation was when this dynamic ended up suspending the rpc server. How did you find out? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From hfventer@REDACTED Thu Jul 28 13:04:42 2011 From: hfventer@REDACTED (Heinrich Venter) Date: Thu, 28 Jul 2011 13:04:42 +0200 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: Fun = fun() -> [ Pid ! ok || _ <- lists:seq(1,10000), Pid <- spawn_link(?MODULE, recv, []) ] end. This wont work, because spawn_link does not generate a list :) > Are you running that expression from the shell? I was executing the tests from the shell yes, but if I do the following with a compiled function [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn(fun() -> test_gen:loop() end) || _ <- lists:seq(1,10000)] ] end) || _ <- lists:seq(1,10)]. I still get the same slow performance. It must be related to creation of the fun then. This gets better results [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn(test_gen, loop, []) || _ <- lists:seq(1,10000)] ] end) || _ <- lists:seq(1,10)]. It also shows a bout 45% performance penalty for spawning OTP processes like mad instead of using spawn(Module, Function, Args). On the other hand using spawn(Fun) seems to be 4 times slower. This is more or less in line with what the efficiency guide says. It pays to measure :) Moral of the story: Spawn your workers with spawn(M,F,A) or use OTP processes if you can afford the slight performance hit and gain a lot of convenience. -]-[einrich From jesper.louis.andersen@REDACTED Thu Jul 28 13:17:47 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 28 Jul 2011 13:17:47 +0200 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: On Thu, Jul 28, 2011 at 13:04, Heinrich Venter wrote: > > [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn(fun() -> > test_gen:loop() end) || _ <- lists:seq(1,10000)] ] end) || _ <- > lists:seq(1,10)]. > > I still get the same slow performance. ?It must be related to creation > of the fun then. There are a couple of problems here. * We generate a list (lists:seq(1,10000)) inside the shell and inside timer:tc/1. This time is also measured against us. * We create a fun in the shell which is then interpreted. Try (not tested) -module(foo). -compile(export_all). test() -> L = lists:seq(1, 10000), F = fun() -> test_gen:loop() end, % this can be "eta"-converted to 'fun test_gen:loop/0' but oh well. [timer:tc([spawn(F) || _ <- L]) || _ <- lists:seq(1, 10)]. or similar. And then call this from the shell. Also do the same for the one below. > > This gets better results > [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn(test_gen, loop, []) || > _ <- lists:seq(1,10000)] ] end) || _ <- lists:seq(1,10)]. This guy doesn't create the fun in the shell, so I expect it to be better. > Moral of the story: Spawn your workers with spawn(M,F,A) or use OTP > processes if you can afford the slight performance hit and gain a lot > of convenience. I wouldn't be so sure :) It also worth it to ask erlc to output the BEAM bytecode and look at it. I have a hunch the shell is still haunting your measurements. -- J. From robert.virding@REDACTED Thu Jul 28 13:33:23 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 28 Jul 2011 13:33:23 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <11834886.317.1311785895222.JavaMail.geo-discussion-forums@yqll11> <2A1AE3F5-E761-4280-85D4-CAD51C8CB15B@erlang-solutions.com> Message-ID: <4E314903.3090407@erlang-solutions.com> On 28/07/2011 12:37, Jesper Louis Andersen wrote: > On Thu, Jul 28, 2011 at 04:28, Jon Watte wrote: > >> All modern languages have a simple hash map class that is efficient and >> built-into the language. JavaScript, Python, Ruby, etc. >> In Erlang, we have dict, and gb_tree, and proplist, and ets, but none of >> them are as easy to use, and some of them have performance problems once you >> scale over a few dozen or a few hundred elements, and others (ets) are way >> overkill for this particular use case. >> >> Or would anyone want to argue that "dict" is actually as good/productive as >> JavaScript objects or Python dicts? > JavaScript, Python, Ruby, etc almost all have ephemeral dict > structures. In Erlang, the structure must be persistent, like in > Clojure/Haskell and so on. This rules out a number of possible > structures. > > As it stands, "dict" is probably the right replacement, with gb_trees > for some situations as they have different speed in those. The "dict" > does fairly well, but it is implemented in Erlang and fast as it is, > it cannot compete with a C-implementation of a ephemeral structure. > > Should we want to BIF/NIFify such a structure, I'd recommend either a > patricia hash table or a HAMT. In fact, that would be an interesting > thing to try. Though I wonder how much I can alter the in-memory > structure of the tree from Erlang terms - I'd really like some cache > efficient packing there for tree nodes. dict uses linear hashing (as does ets) to make sure that there are no long pauses to interrupt the interactivity of the system. This would definitely be more efficient if it were to be done in C in a NIF. Unfortunately you would still have to copy parts of the tree when doing updates to preserve the immutability property of erlang data. Or use a smart way of doing destructive updates and keeping change lists. Robert -- Robert Virding, Erlang Solutions Ltd. From ulf.wiger@REDACTED Thu Jul 28 13:56:00 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 13:56:00 +0200 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> Message-ID: On 28 Jul 2011, at 13:03, Joel Reymont wrote: > > On Jul 28, 2011, at 12:00 PM, Ulf Wiger wrote: > >> Yeah, we (or mainly, Michal Ptaszek) had reason to dig into this fairly recently, and found that tuning can really make a big difference. > > Tuning what? As I mentioned (indirectly) in that email, the distribution buffer busy limit is now tunable, but it has also been increased, as of R14B01, from 128 KB to 1 MB, so that may well be enough. > >> A particularly interesting fault situation was when this dynamic ended up suspending the rpc server. > > How did you find out? Well, we could see from the crash dumps that it was the rpc server's message queue that caused the OOM, and it also listed as 'suspended' (I believe). Initially, we had enabled a system_monitor on busy_dist_port events. As that helped us identify busy_dist_port as a culprit, we [1] inserted some debug printouts in the VM to increase our understanding. Of course, Tony Rogvall and Rickard Green were also extremely helpful. :) BR, Ulf W [1] Michal Ptaszek, that is. I was an enthusiastic cheerleader and messenger boy, relaying ideas - and having a few of my own. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From arjan@REDACTED Thu Jul 28 14:00:19 2011 From: arjan@REDACTED (Arjan Scherpenisse) Date: Thu, 28 Jul 2011 14:00:19 +0200 Subject: [erlang-questions] Zotonic 0.7.0 released! Message-ID: <4E314F53.60407@scherpenisse.net> Dear Zotonic users and Erlang enthousiasts, After some 5 months of development, I'm very pleased to announce the new release of Zotonic, the Erlang CMS [1]. As usual, the zip file can be downloaded from Google code: http://code.google.com/p/zotonic/downloads/list This release is, again, a major feature release. As you can read in the release notes, we have been very actively developing Zotonic and are happy that we can present a new stable version. Please read on for the release notes. Kind regards, The zotonic team, Arjan Scherpenisse Marc Worrell [1] More info on Zotonic: http://zotonic.com Release 0.7.0, released on 2011-07-28 ------------------------------------- * New core features: Native SMTP support for sending and receiving e-mails in any Zotonic site. We integrated Andrew Thompson's gen_smtp library which allows us to manage outgoing ?nd incoming mails. mod_logging provides a new email log-view for inspecting what mails go in and out. A "zotonic" shell command. The "zotonic.sh" shell command has been replaced by a more generic and more powerful shell command with support for pluggable subcommands. Module repository: Zotonic now supports installing system-wide modules which are not part of the core repository. We have created a place where externally contributed modules can be linked at http://modules.zotonic.com/. Modules registered on that site can be easily installed through the "addsite" subcommand. The default website has been replaced by the notion of "skeleton" sites. The "zotonic addsite" command lets you create a new Zotonic website based on one of the (currently two) website templates. * New modules: ** mod_email_relay Relay received e-mails to an user's email address. Serving as an example for the SMTP functionality, this module looks up a username by the local part of a received e-mail and forwards the mail to the mail address the user configured. ** mod_email_receive Handle received e-mails, notifies email observers depending on a stored mapping of recipient addresses. ** mod_import_csv Fairly generic module for importing CSV files, updating or creating new content on the fly. ** mod_import_wordpress Basic import module for Wordpress WXR file format, allowing you to migrate a Wordpress blog into Zotonic. ** Discontinued modules To make Zotonic more lightweight and remove some of the build dependencies, some infrequently used modules have been removed from the core and moved to their own repository, at http://code.google.com/p/zotonic-modules/. These modules are mod_search_solr, mod_pubsub, mod_slideshow, mod_broadcast, mod_imageclipper, mod_admin_event and mod_calendar. They can still be easily installed with the help of the "zotonic installmodule" command. The mod_emailer module (and its esmtp library) has been removed in favor of the native SMTP sending/receiving capabilities. Each module now also can have its own dependencies by including a "deps" subfolder in the module. This is used for example in the mod_pubsub external module which has the exmpp library as a dependency. * Other minor features ** to_json filter for representing template values as JSON objects ** split filter for splitting a string ** slice filter for manipulating lists ** Added {% raw %}..{% endraw %} support for representing literal code blocks. ** erlydtl: Added possibility to define atoms using backquoted strings. ** admin templates are reorganized, allowing to write admin customizations with less code ** translations of the admin updated and more translations added * Bugfixes Too many bugfixes to list. However, the base system is becoming more stable and this release aims to be a good step towards the 1.0. From joelr1@REDACTED Thu Jul 28 14:07:24 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Jul 2011 13:07:24 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> Message-ID: On Jul 28, 2011, at 12:56 PM, Ulf Wiger wrote: > I was an enthusiastic cheerleader and messenger boy I shall make that my official title. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ulf.wiger@REDACTED Thu Jul 28 14:13:51 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 14:13:51 +0200 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: <414EA6E7-DF0E-4BB1-8884-F697BF22FE34@erlang-solutions.com> On 28 Jul 2011, at 13:04, Heinrich Venter wrote: > It also shows a bout 45% performance penalty for spawning OTP > processes like mad instead of using spawn(Module, Function, Args). > On the other hand using spawn(Fun) seems to be 4 times slower. This > is more or less in line with what the efficiency guide says. > > It pays to measure :) It's true that using proc_lib:spawn_link() (which is what the OTP behaviours do) does come with some overhead. Specifically, some data is inserted into the process dictionary for debugging support, and a catch is put on the init function so that more informative crash reports can be produced. We should also remember that raw spawn_link/3 is a very cheap function, so in absolute terms, even spawning OTP processes is a very cheap function. The overhead is a constant factor, and usually well worth it. Then again, if you are writing a pmap, or some other code where minimising overhead is more important than good diagnostics, I agree that one shouldn't uncritically use OTP behaviours. Still, I'm suspicious of the difference between spawn_link/1 and spawn_link/3. When I tried myself on R14B03, I saw no significant difference. Here is the code I used: -module(spf). -export([t1/1, t2/1]). -export([do_recv/0]). t1(N) when N > 0 -> spawn_link(fun() -> receive _ -> ok end end) ! ok, t1(N-1); t1(0) -> ok. t2(N) when N > 0 -> spawn_link(?MODULE, do_recv, []) ! ok, t1(N-1); t2(0) -> ok. do_recv() -> receive _ -> ok end. Sample shell output: Eshell V5.8.4 (abort with ^G) 1> c(spf). {ok,spf} 2> timer:tc(spf,t1,[10000]). {68349,ok} 3> timer:tc(spf,t1,[10000]). {71315,ok} 4> timer:tc(spf,t1,[10000]). {68363,ok} 5> timer:tc(spf,t1,[10000]). {70570,ok} 6> timer:tc(spf,t1,[10000]). {73481,ok} 7> timer:tc(spf,t1,[10000]). {68041,ok} 8> timer:tc(spf,t1,[10000]). {79669,ok} 9> timer:tc(spf,t2,[10000]). {68404,ok} 10> timer:tc(spf,t2,[10000]). {64816,ok} 11> timer:tc(spf,t2,[10000]). {70428,ok} 12> timer:tc(spf,t2,[10000]). {74202,ok} Running the commands too quickly is likely to pollute the measurement, since there is already a "tail" on each run. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From carlsson.richard@REDACTED Thu Jul 28 14:25:40 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 28 Jul 2011 15:25:40 +0300 Subject: [erlang-questions] OTP process startup vs spawn overhead In-Reply-To: References: Message-ID: <4E315544.4000309@gmail.com> On 07/28/2011 02:04 PM, Heinrich Venter wrote: > I was executing the tests from the shell yes, but if I do the > following with a compiled function > > [ timer:tc(fun() -> [ Pid ! ok || Pid<- [spawn(fun() -> > test_gen:loop() end) || _<- lists:seq(1,10000)] ] end) || _<- > lists:seq(1,10)]. > > I still get the same slow performance. It must be related to creation > of the fun then. > > This gets better results > [ timer:tc(fun() -> [ Pid ! ok || Pid<- [spawn(test_gen, loop, []) || > _<- lists:seq(1,10000)] ] end) || _<- lists:seq(1,10)]. > > It also shows a bout 45% performance penalty for spawning OTP > processes like mad instead of using spawn(Module, Function, Args). > On the other hand using spawn(Fun) seems to be 4 times slower. This > is more or less in line with what the efficiency guide says. > > It pays to measure :) > > Moral of the story: Spawn your workers with spawn(M,F,A) or use OTP > processes if you can afford the slight performance hit and gain a lot > of convenience. It also pays to know *what* you measure. In both variant above, your code executes the call timer:tc(fun () -> ...end) ten times. The fun-object is created and passed to timer:tc/1 before the timing starts, so its construction is not part of the measurement. So far so good. However, within each measurement you call spawn() 10 000 times, and in your first example, for each of these times you create a fresh fun-object `fun() -> test_gen:loop() end' despite the fact that this subexpression is actually constant and could be lifted out of the loop, like this: F = fun() -> test_gen:loop() end, [ timer:tc(fun() -> [ Pid ! ok || Pid <- [spawn(F) || _<-lists:seq(1,10000)] ] end) || _<-lists:seq(1,10)]. In your version, you pay the cost both of allocating and instantiating 10 000 fun-objects as well as of garbage collecting these objects (since they immediately become garbage). The moral should be: if you have a tight loop and really want to cut down on the amount of time spent on each iteration - see first if you can move out some code that doesn't actually have to be computed within the loop in the first place - creating a temporary tuple in the middle of the loop should have about the same overhead as creating a fun. /Richard From freza@REDACTED Thu Jul 28 15:20:53 2011 From: freza@REDACTED (Jachym Holecek) Date: Thu, 28 Jul 2011 14:20:53 +0100 Subject: [erlang-questions] callcc or CPS in Erlang? In-Reply-To: References: Message-ID: <20110728132053.GA1081@hanele> # Jon Watte 2011-07-28: > It sounds like what you want is just a way to create a closure, not "CPS" per se. Continuation > Passing Style is a specific approach to slice workloads into separate function invocations; I suppose you're aware of this, but for precision's sake: the point of CPS is to convert each and every instance of each and every form of control transfer into a tail function call -- it's neccessarily a global transformation. What this thread seems to be about would better be described as "callback-driven" approach, or "CPS inspired" perhaps -- not CPS itself. > one use of which is to implement a poor man's threading system. Yes, emulating scheduler activities manually is indeed poor man's threading. There's nothing poor about using real CPS as a foundation of concurrency implementation (Isn't this what at least GHC's LLVM backend is doing? And it sure is precisely how things work in Gambit-C Scheme.) -- though I'm sure there are challenges along the way the extent of which I can't really appreciate. As you note, concurrency is just one use of CPS. On the other hand, one can implement perfectly good first-class continuations without CPS (even in C actually -- makecontext(3) and friends -- which is about what N:M thread implementations must have been doing under the cover). Anyway, could not resist... BR, -- Jachym From sumasai.shivaprasad@REDACTED Thu Jul 28 16:09:58 2011 From: sumasai.shivaprasad@REDACTED (Suma Shivaprasad) Date: Thu, 28 Jul 2011 19:39:58 +0530 Subject: [erlang-questions] Http client sending http response to wrong handler In-Reply-To: References: Message-ID: Hi , Can anyone please let me know if I can work around this issue by changing some parameters on HTTP client or we need a fix for http client for this.? Thanks Suma On Thu, Jul 14, 2011 at 6:55 AM, Jon Watte wrote: > I know this is not what you want to hear, but I ended up writing my own > http client :-/ > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Wed, Jul 13, 2011 at 5:21 AM, Suma Shivaprasad < > sumasai.shivaprasad@REDACTED> wrote: > >> Hi, >> >> Was busy with some other stuff and could not work on this earlier. >> Tried with inets - 5.6 latest version as well and see the same problem >> there as well. >> Now I am able to isolate the cause for this issue. There seems to be a bug >> with the timeout handling code in http client. >> >> Will try to explain the issue - >> >> 1. Lets say we send request R1 at time T1 >> >> 2. At time T2 = T1 + 60 secs, R1 errors out with {error, timeout} from >> http client >> >> 3. At time T3 from the same process we send Request R2 . >> >> 4. At time T4 we get a response for R2 with the R1's request id and R1's >> response data/headers which was supposed to have timed out at T2 and >> cleared from the request table. I could figure this out from the Request >> ids given by http client for R1 and R2. This is causing a cascading failure >> of all subsequent requests from the same process. All requests after a >> timeout from http client are failing >> >> Can you pls let me know if I can work around this by trying out something >> on my end or does it require a fix on http client? >> >> Thanks >> Suma >> >> >> >> >> >> >> >> On Tue, Jun 7, 2011 at 9:19 PM, Ingela Andin wrote: >> >>> Hi! >>> >>> Do you still get this using inets-5.6? In that case we must >>> investigate this further? >>> In such case do you have a minimal example that will reproduce the error? >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> 2011/6/6 Suma Shivaprasad : >>> > Our app makes a lot of HTTP requests and we are facing this issue with >>> both >>> > inets-5.5.1 and 5.3.2. >>> > >>> > Basically our receive clause for the response is trying to match the >>> request >>> > id returned in httpc:request call >>> > and we see that the match always fails with the wrong Request Id . >>> > >>> > We gave seen this mismatch in all 3 receive clauses for >>> > stream_start, stream and stream_end >>> > >>> > What we observed after a lot of trial and error is that if the same >>> pid >>> > makes the http requests , the responses get kind of muddled up but if >>> we >>> > spawn a separate process for the httpc:request , it is better . But >>> here >>> > too we have seen some occurrences of this faulty behaviour. >>> > >>> > Has anyone faced this issue ? >>> > >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Thu Jul 28 16:26:47 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Thu, 28 Jul 2011 17:26:47 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: On Jul 28, 2011, at 3:01 AM, Richard O'Keefe wrote: > > > It might be interesting to run some sort of poll to see what currently > unsupported areas people care about most, particularly the people who > would say "I'd like to use Eiffel but I cannot because it does not > have library support for ." You might want to split such a poll in several groups ? one for hackers/enthusiasts/early adopters and one "for the average programmer". Because the results will be quite different. The first group would probably want something like worker pools, various protocol implementations, control structures or what not. The second group would want image manipulation libraries on par with imagemagick, full PCRE support (with unicode!) and a unified database driver a la JDBC. Some of these choices may overlap, but not all of them. =================================== Dmitrii Dimandt dmitrii@REDACTED ------------------------------------------------------------ Erlang in Russian http://erlanger.ru/ TurkeyTPS http://turkeytps.com/ ------------------------------------------------------------ LinkedIn: http://www.linkedin.com/in/dmitriid GitHub: https://github.com/dmitriid -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Jul 28 19:09:30 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Jul 2011 18:09:30 +0100 Subject: [erlang-questions] common test and mnesia Message-ID: I have two sets of test suites both of which use the same Mnesia schema. It seems that the Mnesia directory gets wiped out after each test suite runs. What is the correct way to use Mnesia with common test? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From fritchie@REDACTED Thu Jul 28 19:45:50 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 28 Jul 2011 12:45:50 -0500 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: Message of "Thu, 28 Jul 2011 13:00:45 +0200." <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> Message-ID: <36530.1311875150@snookles.snookles.com> Ulf Wiger wrote: uw> Actually, these changes in R14B01 are relevant: uw> [...] uw> OTP-8912 The distribution buffer busy limit can now be configured at uw> system startup. For more information see the documentation of the uw> erl +zdbbl command line flag. (Thanks to Scott Lystig Fritchie) I would be quite interested to hear if others have used that flag and what values have been helpful or unhelpful. Since I moved to Basho, I haven't been able to get updates "from the field" about how my Former Employer has been using it. (Or they've told me and I've forgotten about it.) Perhaps one of my former colleagues lurking here on the list could mention something? It would also be quite interesting to know if R14's net_kernel can still be deadlocked. (I'd submitted a patch for R13 to fix a nasty one, and the OTP folks reworked it a bit for inclusion into R14.) The rpc server deadlock ... ouch. One trick for the net_kernel patch was to spawn a new process to send the reply: if that short-lived process blocked on a busy TCP port, at least the main server wouldn't be blocked. -Scott From jay@REDACTED Thu Jul 28 20:16:19 2011 From: jay@REDACTED (Jay Nelson) Date: Thu, 28 Jul 2011 11:16:19 -0700 Subject: [erlang-questions] Backpressure Message-ID: > Ulf Wiger wrote: > The problems with Distributed Erlang are related to a heavy-handed backpressure solution, where processes trying to send to the dist_port are simply suspended if the output queue exceeds a given threshold. Does this only apply to distributed erlang? What about punishment of processes communicating within a node? I've seen senders using more time in local comms*. *Details to be revealed next week during the LA Meetup. jay From joelr1@REDACTED Thu Jul 28 21:26:06 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 28 Jul 2011 20:26:06 +0100 Subject: [erlang-questions] common test and mnesia In-Reply-To: References: Message-ID: There's no issue here, it's a problem with a different part of my code. On Jul 28, 2011, at 6:09 PM, Joel Reymont wrote: > I have two sets of test suites both of which use the same Mnesia schema. > > It seems that the Mnesia directory gets wiped out after each test suite runs. > > What is the correct way to use Mnesia with common test? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From ulf.wiger@REDACTED Thu Jul 28 22:10:59 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 28 Jul 2011 22:10:59 +0200 Subject: [erlang-questions] Backpressure In-Reply-To: References: Message-ID: <5201CD92-7925-4C2E-BEC2-DCB1A1129D7F@erlang-solutions.com> On 28 Jul 2011, at 20:16, Jay Nelson wrote: >> Ulf Wiger wrote: > >> The problems with Distributed Erlang are related to a heavy-handed backpressure solution, where processes trying to send to the dist_port are simply suspended if the output queue exceeds a given threshold. > > Does this only apply to distributed erlang? What about punishment of processes communicating within a node? I've seen senders using more time in local comms*. Only Distributed Erlang. Local message passing does come with a form of backpressure too, as senders are punished with extra reductions if the receiver's message queue is big. I think this feature should go away, as it makes the message send more complex from a scalability point of view, but I know others like it? One can probably reduce locking by allowing the queue length value to be somewhat old, for example. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Fri Jul 29 00:06:19 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 28 Jul 2011 23:06:19 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: > The second group would want image manipulation libraries on par with > imagemagick, full PCRE support (with unicode!) and a unified database driver > a la JDBC. I'm not sure if I qualify as "average" or not, but I would really like to have an edbc API that all the various database drivers (incl. 3rd party ones) supported. I toyed with the idea of writing one and doing some compile time code generation to generate mappings to the drivers we use (postgres, mysql, odbc for mssql) but haven't done it yet as none of the team could agree on what it should look like. :) From ok@REDACTED Fri Jul 29 01:45:21 2011 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 29 Jul 2011 11:45:21 +1200 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> Message-ID: <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> One of the things criticised in the blog entry that we've been responding to was that {ok,Foo} = bar(...), {ok,Foo} = ugh(...) is too easy to write (when you really meant, say, Foo0, Foo1). This is a well defined part of the language, and it would not be a good idea to ban it. But how about an optional style warning (and we really need -warn(on | off, [warning...]) directives) whenever a bound variable appears in a pattern on the left of "="? From overminddl1@REDACTED Fri Jul 29 03:19:59 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 28 Jul 2011 19:19:59 -0600 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: It should be possible to make a nice interface that is parsed transformed into optimized code based on the used engine in a config file or something. On Jul 28, 2011 4:06 PM, "Tim Watson" wrote: >> The second group would want image manipulation libraries on par with >> imagemagick, full PCRE support (with unicode!) and a unified database driver >> a la JDBC. > > I'm not sure if I qualify as "average" or not, but I would really like > to have an edbc API that all the various database drivers (incl. 3rd > party ones) supported. I toyed with the idea of writing one and doing > some compile time code generation to generate mappings to the drivers > we use (postgres, mysql, odbc for mssql) but haven't done it yet as > none of the team could agree on what it should look like. :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Jul 29 11:18:00 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 29 Jul 2011 10:18:00 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: On 29 July 2011 02:19, OvermindDL1 wrote: > It should be possible to make a nice interface that is parsed transformed > into optimized code based on the used engine in a config file or something. > That's exactly what I have in mind. In fact, I was thinking of doing this for a logging API first of all (so you can choose between log4erl, error_logger, lager, etc) and then pulling out the common configuration and code generation features into a library/tool and reusing that to build a database connectivity API. I'll post an update if I actually get around to it! :) From watson.timothy@REDACTED Fri Jul 29 11:30:10 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 29 Jul 2011 10:30:10 +0100 Subject: [erlang-questions] How about a new warning? Was: Re: trouble with erlang or erlang is a ghetto In-Reply-To: <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> <0FA93604-ED5D-4B49-BCB6-8C71AB10FA14@gmail.com> <40209543-C9D7-4FEC-A079-EFD0ADE19C66@erlang-solutions.com> <023CD487-4C7B-44B0-9078-04A0C022D4D8@gmail.com> <4BE27691-DA11-4548-952A-1D5F82FA1E63@cs.otago.ac.nz> Message-ID: On 29 July 2011 00:45, Richard O'Keefe wrote: > One of the things criticised in the blog entry that we've been responding to was > that > ? ? ? ?{ok,Foo} = bar(...), > ? ? ? ?{ok,Foo} = ugh(...) > is too easy to write (when you really meant, say, Foo0, Foo1). > > This is a well defined part of the language, and it would not be a good idea to > ban it. ?But how about an optional style warning (and we really need > ? ? ? ?-warn(on | off, [warning...]) > directives) whenever a bound variable appears in a pattern on the left of "="? We would need to be able to set that warning with a very localised scope though. What you are intending in such a match might be *exactly* what the code says - call bar/1 to get a Foo and then call ugh/2 and match (assert) that the resulting tuple contains exactly the same Foo. Things like a session id, (ets) table id and so on are probably examples of this. I won't comment on whether this is good API design or not. A -warn(on | off, [Warning]) sounds lovely, but would only work at module level (or as some option passed in to the compiler) so how would I do (or override) this for an individual line of code or for a single function? What about function level attributes? That might get the scope at which the switch is applied tight enough to be useful: -pragma(no_warnings, [bound_variable_in_match]). fwibble_ardvarks() -> {ok,Foo} = bar(...), {ok,Foo} = ugh(...), %% etc.... Foo. Harder to implement though, as a bigger change to the compiler I'm guessing. It would however, open up lots of other possibilities - imagine how useful function annotations would be for parse_transform writers, or even at runtime if they were preserved in the generated beam! -transactional(required | start_new, [{distributed, false}]). do_something(Data) -> Object = convert(Data), mnesia:write(Object). From icarus.alive@REDACTED Fri Jul 29 13:18:19 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 29 Jul 2011 16:48:19 +0530 Subject: [erlang-questions] newbie web-development advice / guidance Message-ID: Hi, Looking for some suggestions on how to get started with Web development in Erlang. My background is largely C/C++ systems development, and picked up some Erlang, i.e. I can understand most not-too-complex/large Erlang programs given a little time, unless very esoteric style is adopted, and still find it a bit hard to 'think in Erlang' (a decade of procedural and OO thinking often gets in the way). However, with a turn of events, now I need to do some web-development. Started looking at Nitrogen, but finding the syntax very hard. A steep learning curve. I've done some PHP programming a decade back, and found it to be very simple, and this seems almost like a herculean learning exercise. The tutorial, didn't seem to be structured in a beginner friendly way, and there is hardly any alternative tutorial available. So I wonder as to how many people are really using it. I've had a look at Zotonic as well, but not sure if that might be easier / simpler, and also if it'd be too-much work separating out the CMS from the framework, it might be bit too hard for a beginner to make reasonable progress. My preferred approach would be to keep the backend pretty much REST'ish and use heavier frontend (s.a. jQuery) to interact with it, although I've the need for delivering some streaming media, and strong authentication etc. Also, I shall the services to be available on mobile handsets. Would really appreciate if someone can share thoughts, advice, and pointer to good / comprehensive tutorials etc., to make this learning (& adoption) as simple and easy as possible. Alternatively if someone feels that sticking to PHP (which I am slightly more comfortable with) would be a good idea, shall be happy to have that vote of confidence. Icarus From demeshchuk@REDACTED Fri Jul 29 13:52:49 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Fri, 29 Jul 2011 15:52:49 +0400 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: Many people prefer using Erlang just for REST API and internal distribution, and serve any user-visible content using Python/Ruby/PHP, often on top of some framework (Django, Pylons, Rails, etc). For instance, Heroku and Github use Erlang for internal stuff, and serve content using Ruby. At Mochi Media, we use Python for displaying the pages, and use Erlang only as REST applications. I don't claim this way to be more correct than using Nitrogen/Erlydtl/whatever, but the reason why people do that is because they don't like using Erlang for pages generation. So, you can keep searching, but keep in mind that using another language for serving pages isn't that bad. On Fri, Jul 29, 2011 at 3:18 PM, Icarus Alive wrote: > Hi, > > Looking for some suggestions on how to get started with Web > development in Erlang. > My background is largely C/C++ systems development, and picked up some Erlang, > i.e. I can understand most not-too-complex/large Erlang programs given > a little time, > unless very esoteric style is adopted, and still find it a bit hard to > 'think in Erlang' > (a decade of procedural and OO thinking often gets in the way). However, with a > turn of events, now I need to do some web-development. > > Started looking at Nitrogen, but finding the syntax very hard. A steep > learning curve. > I've done some PHP programming a decade back, and found it to be very simple, > and this seems almost like a herculean learning exercise. The > tutorial, didn't seem > to be structured in a beginner friendly way, and there is hardly any > alternative tutorial > available. So I wonder as to how many people are really using it. I've > had a look at > Zotonic as well, but not sure if that might be easier / simpler, and > also if it'd be > too-much work separating out the CMS from the framework, it might be > bit too hard > for a beginner to make reasonable progress. > > My preferred approach would be to keep the backend pretty much REST'ish and use > heavier frontend (s.a. jQuery) to interact with it, although I've the > need for delivering > some streaming media, and strong authentication etc. Also, I shall the > services to be > available on mobile handsets. > > Would really appreciate if someone can share thoughts, advice, and pointer to > good / comprehensive tutorials etc., to make this learning (& > adoption) as simple and > easy as possible. Alternatively if someone feels that sticking to PHP > (which I am slightly > more comfortable with) would be a good idea, shall be happy to have that vote > of confidence. > > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From marc@REDACTED Fri Jul 29 14:08:34 2011 From: marc@REDACTED (Marc Worrell) Date: Fri, 29 Jul 2011 14:08:34 +0200 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> It mostly depends on what kind of web development you want to do. Is it more of a mobile application than a publishing web site? Does is have specific authentication schemes? Etc. Etc. When your project involves a "normal" dynamic website with publishing, users, some interaction etc. then Zotonic might be a good fit. At least it will prevent you re-inventing quite some wheels. It also uses webmachine, so adding your own specific REST interface should be easy. When your project is really more backend like and not using templates (think of the backend for a mobile app) then it is better to make a server with Webmachine (and whatnot) and use Javascript on the user agent (or a native implementation). - Marc (disclaimer: I am one of the Zotonic maintainers) On 29 jul 2011, at 13:52, Dmitry Demeshchuk wrote: > Many people prefer using Erlang just for REST API and internal > distribution, and serve any user-visible content using > Python/Ruby/PHP, often on top of some framework (Django, Pylons, > Rails, etc). For instance, Heroku and Github use Erlang for internal > stuff, and serve content using Ruby. At Mochi Media, we use Python for > displaying the pages, and use Erlang only as REST applications. > > I don't claim this way to be more correct than using > Nitrogen/Erlydtl/whatever, but the reason why people do that is > because they don't like using Erlang for pages generation. > So, you can keep searching, but keep in mind that using another > language for serving pages isn't that bad. > > On Fri, Jul 29, 2011 at 3:18 PM, Icarus Alive wrote: >> Hi, >> >> Looking for some suggestions on how to get started with Web >> development in Erlang. >> My background is largely C/C++ systems development, and picked up some Erlang, >> i.e. I can understand most not-too-complex/large Erlang programs given >> a little time, >> unless very esoteric style is adopted, and still find it a bit hard to >> 'think in Erlang' >> (a decade of procedural and OO thinking often gets in the way). However, with a >> turn of events, now I need to do some web-development. >> >> Started looking at Nitrogen, but finding the syntax very hard. A steep >> learning curve. >> I've done some PHP programming a decade back, and found it to be very simple, >> and this seems almost like a herculean learning exercise. The >> tutorial, didn't seem >> to be structured in a beginner friendly way, and there is hardly any >> alternative tutorial >> available. So I wonder as to how many people are really using it. I've >> had a look at >> Zotonic as well, but not sure if that might be easier / simpler, and >> also if it'd be >> too-much work separating out the CMS from the framework, it might be >> bit too hard >> for a beginner to make reasonable progress. >> >> My preferred approach would be to keep the backend pretty much REST'ish and use >> heavier frontend (s.a. jQuery) to interact with it, although I've the >> need for delivering >> some streaming media, and strong authentication etc. Also, I shall the >> services to be >> available on mobile handsets. >> >> Would really appreciate if someone can share thoughts, advice, and pointer to >> good / comprehensive tutorials etc., to make this learning (& >> adoption) as simple and >> easy as possible. Alternatively if someone feels that sticking to PHP >> (which I am slightly >> more comfortable with) would be a good idea, shall be happy to have that vote >> of confidence. >> >> Icarus >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best regards, > Dmitry Demeshchuk > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Fri Jul 29 14:41:17 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 29 Jul 2011 13:41:17 +0100 Subject: [erlang-questions] Linked-in driver problem Message-ID: <4E32AA6D.3020606@bobcowdery.plus.com> Hi I have a linked-in driver which pretty much follows the example in the Programming Erlang book. This works very well and I've been using it for a while. I now need two separate instances of this so I went the easy route and created another gen-server for the second instance. Both these gen-servers spawn a reader process and make the reader the port owner. When I start up the program which is a full OTP implementation the supervisor kicks in and closes it all down. No error messages. I have discovered that when the second process spawns its reader the close down occurs with the reader of the first process exiting. If I don't spawn the second reader then it stays up and I can send commands from both processes but obviously can't read responses for the second process. I'm obviously doing something that is incorrect here but I can't figure what. Regards Bob From banibrata.dutta@REDACTED Fri Jul 29 15:01:32 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 29 Jul 2011 18:31:32 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: Thanks @Marc, for taking time to reply this this mail. On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell wrote: > It mostly depends on what kind of web development you want to do. > > Is it more of a mobile application than a publishing web site? > One of the avatars in likely to be a mobile application, but it is neither the first one, nor the only one. However, I am not sure I can classify it as a general publishing site, i.e. same content isn't viewed by several people, rather content uploaded 'somehow' (many ways, many sources) is visible only to the user. What the user can view/has viewed, must be held in tight secrecy. Even admins shouldn't be able to view the content or what content users viewed. User can (in-frequently) share some of the content with others, but I'd say that less than 0.1% of the content (figuratively speaking). Does is have specific authentication schemes? > Etc. Etc. > Honestly, haven't given this enough thought. One of the requirements is to ensure privacy of user data, and that of the interaction. I'm yet to figure out the "how" part. > When your project involves a "normal" dynamic website with publishing, > users, some interaction etc. then Zotonic might be a good fit. > At least it will prevent you re-inventing quite some wheels. It also uses > webmachine, so adding your own specific REST interface should be easy. > Zotonic seemed very interesting, based on a superficial read of the site (I admit, haven't downloaded or tried it), but what seemed a bit intimidating from a newbie standpoint was that I couldn't figure out anything in documentation (s.a. a tutorial) on how to not-use-Zotonic-as-a-CMS-but-as-a-framework. For instance how to write an app that leverage some framework-features of Zotonic, but is largely a REST application. I can understand that such usecase may not the priority for the Zotonic team at the moment. When your project is really more backend like and not using templates (think > of the backend for a mobile app) then it is better to make a server with > Webmachine (and whatnot) and use Javascript on the user agent (or a native > implementation). > If the initial access mechanism to the service/application is desktop browser based is there anything that makes the jQuery + JSON + RESTful backend, harder, less suitable, higher complexity, lower performance, difficult to scale (or any other foreseeable challenges), compared to the template based approach ? > On 29 jul 2011, at 13:52, Dmitry Demeshchuk wrote: > > > Many people prefer using Erlang just for REST API and internal > > distribution, and serve any user-visible content using > > Python/Ruby/PHP, often on top of some framework (Django, Pylons, > > Rails, etc). For instance, Heroku and Github use Erlang for internal > > stuff, and serve content using Ruby. At Mochi Media, we use Python for > > displaying the pages, and use Erlang only as REST applications. > > > > I don't claim this way to be more correct than using > > Nitrogen/Erlydtl/whatever, but the reason why people do that is > > because they don't like using Erlang for pages generation. > > So, you can keep searching, but keep in mind that using another > > language for serving pages isn't that bad. > > > > On Fri, Jul 29, 2011 at 3:18 PM, Icarus Alive > wrote: > >> Hi, > >> > >> Looking for some suggestions on how to get started with Web > >> development in Erlang. > >> My background is largely C/C++ systems development, and picked up some > Erlang, > >> i.e. I can understand most not-too-complex/large Erlang programs given > >> a little time, > >> unless very esoteric style is adopted, and still find it a bit hard to > >> 'think in Erlang' > >> (a decade of procedural and OO thinking often gets in the way). However, > with a > >> turn of events, now I need to do some web-development. > >> > >> Started looking at Nitrogen, but finding the syntax very hard. A steep > >> learning curve. > >> I've done some PHP programming a decade back, and found it to be very > simple, > >> and this seems almost like a herculean learning exercise. The > >> tutorial, didn't seem > >> to be structured in a beginner friendly way, and there is hardly any > >> alternative tutorial > >> available. So I wonder as to how many people are really using it. I've > >> had a look at > >> Zotonic as well, but not sure if that might be easier / simpler, and > >> also if it'd be > >> too-much work separating out the CMS from the framework, it might be > >> bit too hard > >> for a beginner to make reasonable progress. > >> > >> My preferred approach would be to keep the backend pretty much REST'ish > and use > >> heavier frontend (s.a. jQuery) to interact with it, although I've the > >> need for delivering > >> some streaming media, and strong authentication etc. Also, I shall the > >> services to be > >> available on mobile handsets. > >> > >> Would really appreciate if someone can share thoughts, advice, and > pointer to > >> good / comprehensive tutorials etc., to make this learning (& > >> adoption) as simple and > >> easy as possible. Alternatively if someone feels that sticking to PHP > >> (which I am slightly > >> more comfortable with) would be a good idea, shall be happy to have that > vote > >> of confidence. > >> > >> Icarus > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > > > -- > > Best regards, > > Dmitry Demeshchuk > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Fri Jul 29 15:06:13 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 29 Jul 2011 18:36:13 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: Amazingly, I am in an eerily similar situation, and I think we'd interacted in the past. I shall be watching this thread carefully. On Fri, Jul 29, 2011 at 4:48 PM, Icarus Alive wrote: > Hi, > > Looking for some suggestions on how to get started with Web > development in Erlang. > My background is largely C/C++ systems development, and picked up some > Erlang, > i.e. I can understand most not-too-complex/large Erlang programs given > a little time, > unless very esoteric style is adopted, and still find it a bit hard to > 'think in Erlang' > (a decade of procedural and OO thinking often gets in the way). However, > with a > turn of events, now I need to do some web-development. > > Started looking at Nitrogen, but finding the syntax very hard. A steep > learning curve. > I've done some PHP programming a decade back, and found it to be very > simple, > and this seems almost like a herculean learning exercise. The > tutorial, didn't seem > to be structured in a beginner friendly way, and there is hardly any > alternative tutorial > available. So I wonder as to how many people are really using it. I've > had a look at > Zotonic as well, but not sure if that might be easier / simpler, and > also if it'd be > too-much work separating out the CMS from the framework, it might be > bit too hard > for a beginner to make reasonable progress. > > My preferred approach would be to keep the backend pretty much REST'ish and > use > heavier frontend (s.a. jQuery) to interact with it, although I've the > need for delivering > some streaming media, and strong authentication etc. Also, I shall the > services to be > available on mobile handsets. > > Would really appreciate if someone can share thoughts, advice, and pointer > to > good / comprehensive tutorials etc., to make this learning (& > adoption) as simple and > easy as possible. Alternatively if someone feels that sticking to PHP > (which I am slightly > more comfortable with) would be a good idea, shall be happy to have that > vote > of confidence. > > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngocdaothanh@REDACTED Fri Jul 29 15:14:41 2011 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 29 Jul 2011 22:14:41 +0900 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: There is a new (and hot!) web development paradigm brought by backbone.js: * The server side only returns JSON data <--- this minimizes the weakness of Erlang * Use backbone.js for everything else (templates etc.) See tutorials and demos: https://github.com/documentcloud/backbone/wiki/Tutorials%2C-blog-posts-and-example-sites Ngoc On Fri, Jul 29, 2011 at 10:06 PM, Banibrata Dutta wrote: > Amazingly, I am in an eerily similar situation, and I think we'd interacted > in the past. I shall be watching this thread carefully. > On Fri, Jul 29, 2011 at 4:48 PM, Icarus Alive > wrote: >> >> Hi, >> >> Looking for some suggestions on how to get started with Web >> development in Erlang. >> My background is largely C/C++ systems development, and picked up some >> Erlang, >> i.e. I can understand most not-too-complex/large Erlang programs given >> a little time, >> unless very esoteric style is adopted, and still find it a bit hard to >> 'think in Erlang' >> (a decade of procedural and OO thinking often gets in the way). However, >> with a >> turn of events, now I need to do some web-development. >> >> Started looking at Nitrogen, but finding the syntax very hard. A steep >> learning curve. >> I've done some PHP programming a decade back, and found it to be very >> simple, >> and this seems almost like a herculean learning exercise. The >> tutorial, didn't seem >> to be structured in a beginner friendly way, and there is hardly any >> alternative tutorial >> available. So I wonder as to how many people are really using it. I've >> had a look at >> Zotonic as well, but not sure if that might be easier / simpler, and >> also if it'd be >> too-much work separating out the CMS from the framework, it might be >> bit too hard >> for a beginner to make reasonable progress. >> >> My preferred approach would be to keep the backend pretty much REST'ish >> and use >> heavier frontend (s.a. jQuery) to interact with it, although I've the >> need for delivering >> some streaming media, and strong authentication etc. Also, I shall the >> services to be >> available on mobile handsets. >> >> Would really appreciate if someone can share thoughts, advice, and pointer >> to >> good / comprehensive tutorials etc., to make this learning (& >> adoption) as simple and >> easy as possible. Alternatively if someone feels that sticking to PHP >> (which I am slightly >> more comfortable with) would be a good idea, shall be happy to have that >> vote >> of confidence. >> >> Icarus >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > regards, > Banibrata > http://www.linkedin.com/in/bdutta > http://twitter.com/edgeliving > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From marc@REDACTED Fri Jul 29 15:15:38 2011 From: marc@REDACTED (Marc Worrell) Date: Fri, 29 Jul 2011 15:15:38 +0200 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: On 29 jul 2011, at 15:01, Banibrata Dutta wrote: > User can (in-frequently) share some of the content with others, but I'd say that less than 0.1% of the content (figuratively speaking). > Zotonic seemed very interesting, based on a superficial read of the site (I admit, haven't downloaded or tried it), but what seemed a bit intimidating from a newbie standpoint was that I couldn't figure out anything in documentation (s.a. a tutorial) on how to not-use-Zotonic-as-a-CMS-but-as-a-framework. For instance how to write an app that leverage some framework-features of Zotonic, but is largely a REST application. I can understand that such usecase may not the priority for the Zotonic team at the moment. As your users mostly don't share information you could: - add specific data models for storing the user's information - use the zotonic cms for the publication of news etc on the public facing part of your system - access your specific models from the template using the Zotonic 'm' model interface in the templates For your users, I would: - store users in the normal zotonic tables, but hide them from each other using access control - use the authentication/facebook/openid/etc modules for authentication > If the initial access mechanism to the service/application is desktop browser based is there anything that makes the jQuery + JSON + RESTful backend, harder, less suitable, higher complexity, lower performance, difficult to scale (or any other foreseeable challenges), compared to the template based approach ? For an initial setup and speed of development it is useful to keep your templates server based. Later you can add a REST (JSON) interface and templating on the user agent, as this might use less bandwidth. You can also stick with the Zotonic (Nitrogen alike) event model with pushed javascript instead of parsing/interpreting JSON on the client side. For more detailed discussions about Zotonic I would recommend the Zotonic users mailinglist http://groups.google.com/group/zotonic-users - Marc From icarus.alive@REDACTED Fri Jul 29 15:16:40 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 29 Jul 2011 18:46:40 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: <9158FB12-40A7-4276-BD18-C7CACF7690B7@worrell.nl> Message-ID: Would appreciate if my thread wasn't hijacked :-)... although gotta agree that our scenarios are quite similar. On Fri, Jul 29, 2011 at 6:31 PM, Banibrata Dutta wrote: > Thanks @Marc, for taking time to reply this this mail. > On Fri, Jul 29, 2011 at 5:38 PM, Marc Worrell wrote: >> >> It mostly depends on what kind of web development you want to do. >> >> Is it more of a mobile application than a publishing web site? In my case, content characteristics are: 1. Always originating from single source 2. Always consumed by single user 3. Content has short lifetime - once consumed, will be archived 4. Each user has several hundreds of content display to her in "newest first" fashion >> >> Does is have specific authentication schemes? >> Etc. Etc. > > Honestly, haven't given this enough thought. One of the requirements is to > ensure privacy of user data, and that of the interaction. I'm yet to figure > out the "how" part. Well in my case it has to be some strong authentication. Storage, archival and access has to conform to HIPAA compliance. So we know we are dealing with EMR type content. > >> >> When your project involves a "normal" dynamic website with publishing, >> users, some interaction etc. then Zotonic might be a good fit. >> At least it will prevent you re-inventing quite some wheels. ?It also uses >> webmachine, so adding your own specific REST interface should be easy. >> >> >> When your project is really more backend like and not using templates >> (think of the backend for a mobile app) then it is better to make a server >> with Webmachine (and whatnot) and use Javascript on the user agent (or a >> native implementation). >> >> On 29 jul 2011, at 13:52, Dmitry Demeshchuk wrote: >> >> > Many people prefer using Erlang just for REST API and internal >> > distribution, and serve any user-visible content using >> > Python/Ruby/PHP, often on top of some framework (Django, Pylons, >> > Rails, etc). For instance, Heroku and Github use Erlang for internal >> > stuff, and serve content using Ruby. At Mochi Media, we use Python for >> > displaying the pages, and use Erlang only as REST applications. >> > >> > I don't claim this way to be more correct than using >> > Nitrogen/Erlydtl/whatever, but the reason why people do that is >> > because they don't like using Erlang for pages generation. >> > So, you can keep searching, but keep in mind that using another >> > language for serving pages isn't that bad. >> > >> > On Fri, Jul 29, 2011 at 3:18 PM, Icarus Alive >> > wrote: >> >> Hi, >> >> >> >> Looking for some suggestions on how to get started with Web >> >> development in Erlang. >> >> My background is largely C/C++ systems development, and picked up some >> >> Erlang, >> >> i.e. I can understand most not-too-complex/large Erlang programs given >> >> a little time, >> >> unless very esoteric style is adopted, and still find it a bit hard to >> >> 'think in Erlang' >> >> (a decade of procedural and OO thinking often gets in the way). >> >> However, with a >> >> turn of events, now I need to do some web-development. >> >> >> >> Started looking at Nitrogen, but finding the syntax very hard. A steep >> >> learning curve. >> >> I've done some PHP programming a decade back, and found it to be very >> >> simple, >> >> and this seems almost like a herculean learning exercise. The >> >> tutorial, didn't seem >> >> to be structured in a beginner friendly way, and there is hardly any >> >> alternative tutorial >> >> available. So I wonder as to how many people are really using it. I've >> >> had a look at >> >> Zotonic as well, but not sure if that might be easier / simpler, and >> >> also if it'd be >> >> too-much work separating out the CMS from the framework, it might be >> >> bit too hard >> >> for a beginner to make reasonable progress. >> >> >> >> My preferred approach would be to keep the backend pretty much REST'ish >> >> and use >> >> heavier frontend (s.a. jQuery) to interact with it, although I've the >> >> need for delivering >> >> some streaming media, and strong authentication etc. Also, I shall the >> >> services to be >> >> available on mobile handsets. >> >> >> >> Would really appreciate if someone can share thoughts, advice, and >> >> pointer to >> >> good / comprehensive tutorials etc., to make this learning (& >> >> adoption) as simple and >> >> easy as possible. Alternatively if someone feels that sticking to PHP >> >> (which I am slightly >> >> more comfortable with) would be a good idea, shall be happy to have >> >> that vote >> >> of confidence. >> >> >> >> Icarus >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > >> > >> > >> > -- >> > Best regards, >> > Dmitry Demeshchuk >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > regards, > Banibrata > http://www.linkedin.com/in/bdutta > http://twitter.com/edgeliving > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From marc@REDACTED Fri Jul 29 15:19:18 2011 From: marc@REDACTED (Marc Worrell) Date: Fri, 29 Jul 2011 15:19:18 +0200 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: <72C98F3C-AEC9-4810-9F1F-6A0EF4EC4502@worrell.nl> The problem with javascript-for-everything-on-the-user-agent is that your site is not a web site anymore but an application. This makes your site very hard to access for crawlers, screen readers etc. etc. - Marc On 29 jul 2011, at 15:14, Ngoc Dao wrote: > There is a new (and hot!) web development paradigm brought by backbone.js: > * The server side only returns JSON data <--- this minimizes the > weakness of Erlang > * Use backbone.js for everything else (templates etc.) > > See tutorials and demos: > https://github.com/documentcloud/backbone/wiki/Tutorials%2C-blog-posts-and-example-sites > > Ngoc From ngocdaothanh@REDACTED Fri Jul 29 16:05:25 2011 From: ngocdaothanh@REDACTED (Ngoc Dao) Date: Fri, 29 Jul 2011 23:05:25 +0900 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: <72C98F3C-AEC9-4810-9F1F-6A0EF4EC4502@worrell.nl> References: <72C98F3C-AEC9-4810-9F1F-6A0EF4EC4502@worrell.nl> Message-ID: Yes. That's a different paradigm. But if your site is behind a login page (like Facebook, Gmail) then that's OK. Anyway, that's not your problem, that's the problem of crawlers, screen readers etc. etc. Ngoc On Fri, Jul 29, 2011 at 10:19 PM, Marc Worrell wrote: > The problem with javascript-for-everything-on-the-user-agent is that your site is not a web site anymore but an application. > This makes your site very hard to access for crawlers, screen readers etc. etc. > > - Marc > > > On 29 jul 2011, at 15:14, Ngoc Dao wrote: > >> There is a new (and hot!) web development paradigm brought by backbone.js: >> * The server side only returns JSON data <--- this minimizes the >> weakness of Erlang >> * Use backbone.js for everything else (templates etc.) >> >> See tutorials and demos: >> https://github.com/documentcloud/backbone/wiki/Tutorials%2C-blog-posts-and-example-sites >> >> Ngoc > > From overminddl1@REDACTED Fri Jul 29 16:13:00 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Fri, 29 Jul 2011 08:13:00 -0600 Subject: [erlang-questions] newbie web-development advice / guidance Message-ID: For note, my background is also 15+ years of C++, and although it and Erlang are different, C++ templates are very much similar to Erlangs syntax (and dang near identical to Haskell), so if you were a heavy template writer like I am then think of it that way. I jumped into Erlang and picked it up very quickly. Oh, and I use Nitrogen, mostly as I like to generate pages in code and theme and layout using css; I try to use as little straight html as possible, but using html directly is something Zotonic if good at if you prefer that. I can help with your C++ to Erlang mental conversions if you want. On Jul 29, 2011 5:18 AM, "Icarus Alive" wrote: -------------- next part -------------- An HTML attachment was scrubbed... URL: From wsongcn@REDACTED Fri Jul 29 16:27:09 2011 From: wsongcn@REDACTED (Andy W. Song) Date: Fri, 29 Jul 2011 22:27:09 +0800 Subject: [erlang-questions] yet another Erlang WebSocket implementation Message-ID: Hi, Not to reinvent the wheel, just as an exercise for learning Erlang. Here you go. Copied some ideas from this project as well as misultin and cowboy. It is not complete right now. Doesn't support segment yet, which I hope is not used very often. And it should be fairly easy to replace the current implementation in misultin. Enjoy. Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Fri Jul 29 16:27:54 2011 From: dan@REDACTED (Daniel Dormont) Date: Fri, 29 Jul 2011 10:27:54 -0400 Subject: [erlang-questions] what does l/1 really do? Message-ID: While reading through some old discussion here or on TrapExit when I was first trying to understand hot code loading, I ran across the shell command l/1. Since then I've used it to good effect in patching a live system (Ejabberd, specifically) with new or upgraded modules. But I'd like to understand a little better what it's doing behind the scenes. I didn't see any mention of it in the manual, and it's a little hard to Google for single letters :), so I thought I'd ask here. I also have a more specific question. I have a whole bunch of gen_fsm processes that I *believe* have a common supervisor but I won't swear to it. If I want to upgrade the module, is there a way to institute the code change right away across all those processes? thanks, Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Fri Jul 29 17:02:50 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 29 Jul 2011 17:02:50 +0200 Subject: [erlang-questions] what does l/1 really do? In-Reply-To: References: Message-ID: l/1 do eaxctly l(Mod) -> code:purge(Mod), code:load_file(Mod). See https://github.com/erlang/otp/blob/dev/lib/stdlib/src/c.erl#L251 On Fri, Jul 29, 2011 at 4:27 PM, Daniel Dormont wrote: > While reading through some old discussion here or on TrapExit when I was > first trying to understand hot code loading, I ran across the shell command > l/1. Since then I've used it to good effect in patching a live system > (Ejabberd, specifically) with new or upgraded modules. But I'd like to > understand a little better what it's doing behind the scenes. I didn't see > any mention of it in the manual, and it's a little hard to Google for single > letters :), so I thought I'd ask here. > > I also have a more specific question. I have a whole bunch of gen_fsm > processes that I *believe* have a common supervisor but I won't swear to it. > If I want to upgrade the module, is there a way to institute the code change > right away across all those processes? > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From hm@REDACTED Fri Jul 29 17:36:25 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 29 Jul 2011 17:36:25 +0200 Subject: [erlang-questions] what does l/1 really do? In-Reply-To: References: Message-ID: http://www.erlang.org/doc/man/c.html#l-1 /H?kan On Fri, Jul 29, 2011 at 4:27 PM, Daniel Dormont wrote: > While reading through some old discussion here or on TrapExit when I was > first trying to understand hot code loading, I ran across the shell command > l/1. Since then I've used it to good effect in patching a live system > (Ejabberd, specifically) with new or upgraded modules. But I'd like to > understand a little better what it's doing behind the scenes. I didn't see > any mention of it in the manual, and it's a little hard to Google for single > letters :), so I thought I'd ask here. From joelr1@REDACTED Fri Jul 29 18:12:50 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 29 Jul 2011 17:12:50 +0100 Subject: [erlang-questions] rebar and compiling common test suites Message-ID: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> I have common test suites under test/ and have added {src_dirs, ["test"]} to rebar.config. Still, Erlang code under test/ is not compiled when I run 'rebar compile'. Is there a way to fix this? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From watson.timothy@REDACTED Fri Jul 29 19:58:49 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 29 Jul 2011 18:58:49 +0100 Subject: [erlang-questions] rebar and compiling common test suites In-Reply-To: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> References: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> Message-ID: The way rebar's common_test support works, it takes advantage of the fact that common_test automatically compiles test suites (and helper modules) for you. So the rebar compiler isn't normally used for ct support. I'm assuming that you're setting src_dirs to include "test" because you want to compile them *manually* for some reason - is this essential? I might be able to suggest where to dig around in the rebar code to figure out why these modules are being excluded from compile (despite the "test" dir being included in src_dirs) but I don't fancy digging around if you're just unaware that `rebar ct` "just works" - let me know eh. Cheers, Tim On 29 July 2011 17:12, Joel Reymont wrote: > I have common test suites under test/ and have added {src_dirs, ["test"]} to rebar.config. > > Still, Erlang code under test/ is not compiled when I run 'rebar compile'. > > Is there a way to fix this? > > ? ? ? ?Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From icarus.alive@REDACTED Fri Jul 29 20:17:05 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Fri, 29 Jul 2011 23:47:05 +0530 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: On Fri, Jul 29, 2011 at 7:43 PM, OvermindDL1 wrote: > For note, my background is also 15+ years of C++, and although it and Erlang > are different, C++ templates are very much similar to Erlangs syntax (and > dang near identical to Haskell), so if you were a heavy template writer like > I am then think of it that way.? I jumped into Erlang and picked it up very > quickly. @OvermindDL1, very glad to hear that. Not familiar with Haskell, but C++ templates are a familiar territory. > Oh, and I use Nitrogen, mostly as I like to generate pages in code and theme > and layout using css; I try to use as little straight html as possible, but > using html directly is something Zotonic if good at if you prefer that. > > I can help with your C++ to Erlang mental conversions if you want. That would be really a great help. However, if you (or anyone else here) had a chance to compare Nitrogen & Zotonic on the developer efficiency and learning-curve aspects, would be good to hear. Also, how does it compare in performance terms, i.e. programmatic HTML generation in Erlang code, versus template driven HTML generation using a dedicated templating engine (if I understood it correctly). > > On Jul 29, 2011 5:18 AM, "Icarus Alive" wrote: > From dan@REDACTED Fri Jul 29 21:52:56 2011 From: dan@REDACTED (Daniel Dormont) Date: Fri, 29 Jul 2011 15:52:56 -0400 Subject: [erlang-questions] what does l/1 really do? In-Reply-To: References: Message-ID: 2011/7/29 H?kan Mattsson > http://www.erlang.org/doc/man/c.html#l-1 > > Good to know. In my defense, I couldn't find anything in the manual that mentions "c" as the module to look in for those functions... Any thoughts on my other question: if there are a variable-sized set of dynamically-launched gen_fsm's using a particular module, assuming most/all of them are idle or will be momentarily, is there a recommended way to propagate a code upgrade across all of them? dan > /H?kan > > On Fri, Jul 29, 2011 at 4:27 PM, Daniel Dormont > wrote: > > While reading through some old discussion here or on TrapExit when I was > > first trying to understand hot code loading, I ran across the shell > command > > l/1. Since then I've used it to good effect in patching a live system > > (Ejabberd, specifically) with new or upgraded modules. But I'd like to > > understand a little better what it's doing behind the scenes. I didn't > see > > any mention of it in the manual, and it's a little hard to Google for > single > > letters :), so I thought I'd ask here. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skruger@REDACTED Fri Jul 29 22:22:16 2011 From: skruger@REDACTED (Shaun Kruger) Date: Fri, 29 Jul 2011 14:22:16 -0600 (MDT) Subject: [erlang-questions] what does l/1 really do? In-Reply-To: Message-ID: <84251990-6369-4823-81e8-240efc0dc416@quadratic> If I understand things correctly then simply loading the module should do it. I hope someone corrects me if I'm wrong, but I believe that calling a fully qualified Module:Function() causes the latest code to be executed. The OTP behaviours all do this fully qualified kind of call to get to the callbacks they use. My experience in my projects is that I never have to do anything special before the newest version of code gets executed. It is my expectation that OTP does something special to synchronize a code change for an OTP release upgrade so that all the OTP behaviour based modules stop executing briefly before the code upgrade happens. If you aren't using those release upgrade facilities then the newest code gets executed the next time control passes to your module that implements the gen_fsm behaviour. This has bit me a number of times in my development enviornment when I change the fields in a record and my module dies on a badrecord exception one short moment after I load the new version of code. Shaun ----- Original Message ----- > From: "Daniel Dormont" > To: "H?kan Mattsson" > Cc: "erlang-questions" > Sent: Friday, July 29, 2011 1:52:56 PM > Subject: Re: [erlang-questions] what does l/1 really do? > > > > 2011/7/29 H?kan Mattsson < hm@REDACTED > > > > http://www.erlang.org/doc/man/c.html#l-1 > > > > Good to know. In my defense, I couldn't find anything in the manual > that mentions "c" as the module to look in for those functions... > > Any thoughts on my other question: if there are a variable-sized set > of dynamically-launched gen_fsm's using a particular module, > assuming most/all of them are idle or will be momentarily, is there > a recommended way to propagate a code upgrade across all of them? > > dan > > > /H?kan > > > On Fri, Jul 29, 2011 at 4:27 PM, Daniel Dormont > < dan@REDACTED > wrote: > > > > > While reading through some old discussion here or on TrapExit when > > I was > > first trying to understand hot code loading, I ran across the shell > > command > > l/1. Since then I've used it to good effect in patching a live > > system > > (Ejabberd, specifically) with new or upgraded modules. But I'd like > > to > > understand a little better what it's doing behind the scenes. I > > didn't see > > any mention of it in the manual, and it's a little hard to Google > > for single > > letters :), so I thought I'd ask here. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From joelr1@REDACTED Fri Jul 29 22:56:13 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 29 Jul 2011 21:56:13 +0100 Subject: [erlang-questions] rebar and compiling common test suites In-Reply-To: References: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> Message-ID: <6E634312-5985-4660-A06B-21C7784FA144@gmail.com> On Jul 29, 2011, at 6:58 PM, Tim Watson wrote: > The way rebar's common_test support works, it takes advantage of the > fact that common_test automatically compiles test suites (and helper > modules) for you. Compiling CT suites and related code as part of running CT takes too long. Often, you need to check CT logs to figure out just where you made an error or typo. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joel.meyer@REDACTED Fri Jul 29 23:14:18 2011 From: joel.meyer@REDACTED (Joel Meyer) Date: Fri, 29 Jul 2011 14:14:18 -0700 Subject: [erlang-questions] Linked-in driver problem In-Reply-To: <4E32AA6D.3020606@bobcowdery.plus.com> References: <4E32AA6D.3020606@bobcowdery.plus.com> Message-ID: Take a look at http://www.erlang.org/doc/man/erl_driver.htmland ERL_DRV_FLAG_USE_PORT_LOCKING. My guess is that you've got driver locking right now and it's failing when you try to acquire the lock again for the second port. Cheers, Joel On Fri, Jul 29, 2011 at 5:41 AM, Bob Cowdery wrote: > Hi > > I have a linked-in driver which pretty much follows the example in the > Programming Erlang book. This works very well and I've been using it for > a while. I now need two separate instances of this so I went the easy > route and created another gen-server for the second instance. Both these > gen-servers spawn a reader process and make the reader the port owner. > > When I start up the program which is a full OTP implementation the > supervisor kicks in and closes it all down. No error messages. I have > discovered that when the second process spawns its reader the close down > occurs with the reader of the first process exiting. If I don't spawn > the second reader then it stays up and I can send commands from both > processes but obviously can't read responses for the second process. I'm > obviously doing something that is incorrect here but I can't figure what. > > Regards > Bob > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Fri Jul 29 23:19:04 2011 From: mihai@REDACTED (Mihai Balea) Date: Fri, 29 Jul 2011 17:19:04 -0400 Subject: [erlang-questions] Gproc in a dynamic cluster Message-ID: <3E83ACB7-9B81-4812-AF4A-F153D2CFA3F8@hates.ms> Hi all, Here's a scenario: - start Node1 - start gproc in global mode on node 1 (via {gproc, [{gproc_dist, all}]} in app.config) - start Node2 - start gproc in global mode on node 2 (same way) - connect the two nodes in a cluster, via net_adm:ping/1 The two nodes see each other, as expected, but the two instances of gproc don't seem to be aware of each other's presence - global registrations done on one node are not visible on the other side. Am I missing something? Is there a way to make gproc aware of new nodes in a cluster, either automatically or manually? Thanks in advance, Mihai From joelr1@REDACTED Fri Jul 29 23:21:57 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 29 Jul 2011 22:21:57 +0100 Subject: [erlang-questions] common test cleanup Message-ID: <6EB20164-6F29-43BD-A595-75E2047EE4F6@gmail.com> Consider the following fragment? end_per_testcase(pass_config, Config) -> {_, Config1} = ?config(saved_config, Config), true = ?config(pass_config, Config1). pass_config(Config) -> {save_config, [{pass_config, true}|Config]}. This results in a message "Could not find element saved_config in Config." and a crash. Is there no way to modify the Config in the test function and pass it to end_per_testcase? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From alex.arnon@REDACTED Fri Jul 29 23:27:53 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Sat, 30 Jul 2011 00:27:53 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: I had built a rather basic prototype that uses a jnode running a JDBC proxy. This approach would increase latency, but provide access to every RDBMS under the sun. I think that what is truly missing is an agreed-upon, standard EDBC API definition. IMHO one that is based on JDBC could certainly be good enough. Once that is in place, implementation is just a detail :) Erlang Community, time to claim your place in the Enterprise!!!! :) On Fri, Jul 29, 2011 at 12:18 PM, Tim Watson wrote: > On 29 July 2011 02:19, OvermindDL1 wrote: > > It should be possible to make a nice interface that is parsed transformed > > into optimized code based on the used engine in a config file or > something. > > > > That's exactly what I have in mind. In fact, I was thinking of doing > this for a logging API first of all (so you can choose between > log4erl, error_logger, lager, etc) and then pulling out the common > configuration and code generation features into a library/tool and > reusing that to build a database connectivity API. I'll post an update > if I actually get around to it! :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Jul 30 02:44:52 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 30 Jul 2011 01:44:52 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: On 29 July 2011 22:27, Alex Arnon wrote: > I had built a rather basic prototype that uses a jnode running a JDBC proxy. > This approach would increase latency, but provide access to every RDBMS > under the sun. That really scares me, and I would hesitate to use it. If you planning on conquering the enterprise, I suspect that jumping out of you main language VM into another one just to do database access isn't going to go down too well. Although I could be wrong, because honestly I've seen big companies buy "All Kinds of Stuff" (TM). Once I'd gone to that much trouble, I might as well write the whole shebang in Java. In fact, I prefer picking "the right tool for the job" for each part of a project and Erlang doesn't fit every niche so sometimes turning to Java, Python and other things is the correct choice. Erlang does have some decent open source libraries for doing database access (to postgres at least) and I've seen a very good closed source Oracle driver written as a port_driver using the OCL libraries - quick and stable, but sadly not available to the general public. The author claims he'll help do a rewrite when native processes get added to Erlang. :) > > I think that what is truly missing is an agreed-upon, standard EDBC API > definition. IMHO one that is based on JDBC could certainly be good enough. > Once that is in place, implementation is just a detail :) > Erlang Community, time to claim your place in the Enterprise!!!! :) > I kind of agree that a standard API would be good. Unless it comes from Ericsson or gets into the OTP distro, it's not really going to be a standard though. If someone starts one, they really need to get on this list and push people for feedback. I'd also suggest making it as thin a veneer as possible on top of the real library/application being used, as this is less likely to introduce problems. I think a big part of the reason why medium/large enterprises don't tend to use Erlang as a core development language, actually has nothing to do with technology. It's a resourcing issue. It's harder to find real Erlang expertise (i.e., commercial experience) than it is to find Java and/or .NET developers and that increases both cost and risk. I certainly think that Erlang's productivity and other factors (like its built-in support for making fault tolerant applications) can actually outweigh the initial cost of hiring good developers, but again this is a risk that many big businesses (which are mainly run by accountants in my experience) won't take. Start-ups and niche sectors are a completely different kettle of fish. Finally, I agree that JDBC has some good patterns to follow, but it's whole design is very Object Oriented (i.e., mutable state oriented) and therefore not much of it is really a good fit for Erlang. Things that might translate nicely as features are (IMHO): 1. Runtime discovery of drivers 2. Runtime discovery of data source(s) a la JNDI - this is very useful in Java applications and something like it (maybe gproc based?) would be nice 3. Standard API for dealing with connections (and possibly connection pools, although I'm not sure I'd want to open that particular can of worms myself) 4. Standard API for working with database metadata (e.g., INFORMATION_SCHEMA, system views and the like) 5. Standard API for working with result sets (e.g., moving/scrolling, indexed access to columns in the current row, etc) 6. QLC queries for interacting with result sets? Not sure if that last one is really a good idea - it's very late here and I've had a busy week. AFAICT good API design for Erlang doesn't usually involve exposing internal data structures (records, etc) to the end user and any state is usually hidden as much as possible. My assumption is that the various API modules involved would usually just return an opaque handle to the user, which is then used during further calls. I certainly wouldn't go off using parameterised modules as they're not supported. So you'd get something more like: {ok, DSHandle} = edbc_datasource:get_named_datasource(postgres_test), {ok, ConnHandle} = edbc_connection:open(DSHandle, [PerConnOptions]), {ok, TxnHandle} = edbc_transaction:begin(ConnHandle, [{isolation, read_committed]), try %% fall back to normal (not prepared) statement when this doesn't work... {ok, StatementHandle} = ebdc_connection:prepare_statement(ConnHandle, SQL), {ok, StatementHandle} = edbc_statement:bind_named_params([{id, 12345}, {customer_name, "Mr Smith"}]), {ok, Cursor} = edbc_connection:execute_query(ConnHandle, StatementHandle), edbc_rowset:foreach(fun(Row, Col, Value) -> io:format("~p:~p = ~p~n", [Row, Col, Value]) end, Cursor), %% version of execute/2 that takes raw SQL instead of prepared statements edbc_connection:execute_update(ConnHandle, "INSERT INTO Food VALUES ('Ice Cream', 'Vanilla')"), {ok, committed} = edbc_transaction:commit(TxnHandle) catch _:_ -> {ok, aborted} = edbc_transaction:rollback(TxnHandle) after {ok, closed} = edbc_connection:close(ConnHandle) Probably a lot of these calls to specific modules could be hidden behind a slightly higher level API: {ok, Conn} = edbc:open_connection(postgres_test), Results = edbc:map(Conn, fun do_with_each_cell/2, SqlQuery), %% or Tree = edbc:fold(Conn, Query, fun(Row, Col, Val, Acc) -> gb_trees:enter({Row, Col}, Val, Acc) end), etc..... You're right that having a good standard API against which all of the various implementations could be used is a very nice idea. The question is, how close are the existing libraries/applications to a *nice* API (whatever that is) and how much work is involved in bridging between them. This is why I'd like to see if the binding between API and implementation can be generated - I'd like to avoid having state in the API and let the underlying implementations deal with their own states, servers, errors, etc. From watson.timothy@REDACTED Sat Jul 30 02:55:38 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 30 Jul 2011 01:55:38 +0100 Subject: [erlang-questions] rebar and compiling common test suites In-Reply-To: <6E634312-5985-4660-A06B-21C7784FA144@gmail.com> References: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> <6E634312-5985-4660-A06B-21C7784FA144@gmail.com> Message-ID: Ok Joel, how are you configuring src_dirs? It should be part of erl_opts: {erl_opts, [no_debug_info, {src_dirs, ["test"]} | MoreOpts]}. That works for me. On 29 July 2011 21:56, Joel Reymont wrote: > > On Jul 29, 2011, at 6:58 PM, Tim Watson wrote: > >> The way rebar's common_test support works, it takes advantage of the >> fact that common_test automatically compiles test suites (and helper >> modules) for you. > > Compiling CT suites and related code as part of running CT takes too long. > > Often, you need to check CT logs to figure out just where you made an error or typo. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > From sam@REDACTED Sat Jul 30 13:39:28 2011 From: sam@REDACTED (Sam Elliott) Date: Sat, 30 Jul 2011 12:39:28 +0100 Subject: [erlang-questions] what does l/1 really do? In-Reply-To: References: Message-ID: Hi Dan, In the supervisor's child specification, the last element of the tuple should be an array of atoms, which are the names of modules to get reloaded when an update is done (i think). to quote the documentation: "Modules [the array] is used by the release handler during code replacement to determine which processes are using a certain module. As a rule of thumb Modules should be a list with one element [Module], where Module is the callback module, if the child process is a supervisor, gen_server or gen_fsm. If the child process is an event manager (gen_event) with a dynamic set of callback modules, Modules should be dynamic. See OTP Design Principles for more information about release handling." (from http://www.erlang.org/doc/man/supervisor.html) I hope that's helpful. Sam -- Sam Elliott sam@REDACTED -- On Fri, Jul 29, 2011 at 3:27 PM, Daniel Dormont wrote: > While reading through some old discussion here or on TrapExit when I was > first trying to understand hot code loading, I ran across the shell command > l/1. Since then I've used it to good effect in patching a live system > (Ejabberd, specifically) with new or upgraded modules. But I'd like to > understand a little better what it's doing behind the scenes. I didn't see > any mention of it in the manual, and it's a little hard to Google for single > letters :), so I thought I'd ask here. > > I also have a more specific question. I have a whole bunch of gen_fsm > processes that I *believe* have a common supervisor but I won't swear to it. > If I want to upgrade the module, is there a way to institute the code change > right away across all those processes? > > thanks, > Dan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dvdougdv@REDACTED Sat Jul 30 16:26:28 2011 From: dvdougdv@REDACTED (doug) Date: Sat, 30 Jul 2011 10:26:28 -0400 Subject: [erlang-questions] R14B03 won't build on debian squeeze Message-ID: Is there something I can do? cd lib && \ ERL_TOP=/home/douglas/erlang PATH=/home/douglas/erlang/bootstrap/bin:${PATH} \ make opt SECONDARY_BOOTSTRAP=true make[1]: Entering directory `/home/douglas/erlang/lib' make[2]: Entering directory `/home/douglas/erlang/lib/hipe' === Entering application hipe make[3]: Entering directory `/home/douglas/erlang/lib/hipe/rtl' (cd ../main && make hipe.hrl) make[4]: Entering directory `/home/douglas/erlang/lib/hipe/main' sed -e "s;%VSN%;3.8;" ../../hipe/main/hipe.hrl.src > ../../hipe/main/hipe.hrl make[4]: Leaving directory `/home/douglas/erlang/lib/hipe/main' erlc -W +debug_info +inline -o../ebin hipe_rtl.erl exec: 28: /home/douglas/Downloads/otp_src_R14B03/bin/x86_64-unknown-linux-gnu/erlexec: not found make[3]: *** [../ebin/hipe_rtl.beam] Error 2 make[3]: Leaving directory `/home/douglas/erlang/lib/hipe/rtl' make[2]: *** [opt] Error 2 make[2]: Leaving directory `/home/douglas/erlang/lib/hipe' make[1]: *** [opt] Error 2 make[1]: Leaving directory `/home/douglas/erlang/lib' make: *** [secondary_bootstrap_build] Error 2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dvdougdv@REDACTED Sat Jul 30 17:13:05 2011 From: dvdougdv@REDACTED (doug) Date: Sat, 30 Jul 2011 11:13:05 -0400 Subject: [erlang-questions] R14B03 won't build on debian squeeze Message-ID: Please disregard the post. I have it running now. What I did differently I don't know Thank you, Douglas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Jul 30 18:51:52 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 30 Jul 2011 09:51:52 -0700 Subject: [erlang-questions] newbie web-development advice / guidance In-Reply-To: References: Message-ID: The "web" pages I've written in Erlang are mostly just status and management REST interfaces for a system whose main goal is something else, but what I've found so far: If you want a web interface where you take the request once headers are parsed and the request decoded, use mochiweb. If you want an industrial-strength HTTP protocol server stack (things like content type negotiation, etc) for a REST-only implementation, use webmachine. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Jul 29, 2011 at 4:18 AM, Icarus Alive wrote: > Hi, > > Looking for some suggestions on how to get started with Web > development in Erlang. > My background is largely C/C++ systems development, and picked up some > Erlang, > i.e. I can understand most not-too-complex/large Erlang programs given > a little time, > unless very esoteric style is adopted, and still find it a bit hard to > 'think in Erlang' > (a decade of procedural and OO thinking often gets in the way). However, > with a > turn of events, now I need to do some web-development. > > Started looking at Nitrogen, but finding the syntax very hard. A steep > learning curve. > I've done some PHP programming a decade back, and found it to be very > simple, > and this seems almost like a herculean learning exercise. The > tutorial, didn't seem > to be structured in a beginner friendly way, and there is hardly any > alternative tutorial > available. So I wonder as to how many people are really using it. I've > had a look at > Zotonic as well, but not sure if that might be easier / simpler, and > also if it'd be > too-much work separating out the CMS from the framework, it might be > bit too hard > for a beginner to make reasonable progress. > > My preferred approach would be to keep the backend pretty much REST'ish and > use > heavier frontend (s.a. jQuery) to interact with it, although I've the > need for delivering > some streaming media, and strong authentication etc. Also, I shall the > services to be > available on mobile handsets. > > Would really appreciate if someone can share thoughts, advice, and pointer > to > good / comprehensive tutorials etc., to make this learning (& > adoption) as simple and > easy as possible. Alternatively if someone feels that sticking to PHP > (which I am slightly > more comfortable with) would be a good idea, shall be happy to have that > vote > of confidence. > > Icarus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Jul 30 19:04:32 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 30 Jul 2011 10:04:32 -0700 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <03CF0863-2669-4FD9-BEF3-392600D37607@erlang-solutions.com> <1311757664.3458.YahooMailNeo@web111401.mail.gq1.yahoo.com> <4210E7B2-14B4-424C-BE36-720988A0277E@erlang-solutions.com> Message-ID: > > > BEAM is starting to make use of NUMA, for example when allowing you to > control the binding of schedulers to cores. See e.g. > > The real efficiencies come from processes (and their working set) being bound to particular spots in the memory hierarchy, though. I imagine that, because processes have affinity for schedulers, then schedulers being bound to cores would help some, but when the runtime decides to migrate a process, it probably needs to consider how far down the memory hierarchy to "fork" the process. (These days, we have to worry about L1, L2, L3 and NUMA nodes as distinct points in this choice) Similarly, if one queue is falling behind, and another queue on the same core (using hyper-threading) is lightly loaded, it doesn't really make sense to shift load between those two hyper-threads as they are limited on the same functional units and L1. It sounds like you're already considering these things; just making sure it's stated explicitly in this thread! > > > Yes, but one thing I learned while at Ericsson was that NEBS-compliant ATCA > processor boards don't exactly stay on the leading edge of processor > capacity. > That is of course a consideration. On the other hand, I imagine projects run over several years, and what's best price/performance in data centers today will likely trickle down to the telecom industry eventually :-) > > The key, in my experience, is not usually to go as fast as possible, but to > deliver reliable and predictable performance that is good enough for the > problem at hand. As many have pointed out through the years, Erlang was > never about delivering maximum performance. > > > I'm with you there! If I know that I can add another node to get close to linear X% increased capacity, that's really powerful. Sincerely, jw -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Sat Jul 30 23:59:08 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 30 Jul 2011 22:59:08 +0100 Subject: [erlang-questions] rebar and compiling common test suites In-Reply-To: References: <9117446E-82D8-4C15-BB20-359D1F8F80EC@gmail.com> <6E634312-5985-4660-A06B-21C7784FA144@gmail.com> Message-ID: <0047C21F-3E42-4E57-AE74-0A737A663738@gmail.com> On Jul 30, 2011, at 1:55 AM, Tim Watson wrote: > {erl_opts, [no_debug_info, {src_dirs, ["test"]} | MoreOpts]}. Works like a charm, thanks! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From alex.arnon@REDACTED Sun Jul 31 12:44:48 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Sun, 31 Jul 2011 13:44:48 +0300 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: On Sat, Jul 30, 2011 at 3:44 AM, Tim Watson wrote: > On 29 July 2011 22:27, Alex Arnon wrote: > > I had built a rather basic prototype that uses a jnode running a JDBC > proxy. > > This approach would increase latency, but provide access to every RDBMS > > under the sun. > > That really scares me, and I would hesitate to use it. If you planning > on conquering the enterprise, I suspect that jumping out of you main > language VM into another one just to do database access isn't going to > go down too well. Although I could be wrong, because honestly I've > seen big companies buy "All Kinds of Stuff" (TM). > > I assure you, I am not planning on conquering any Enterprises :) However, I disagree with the above... please see below. > Once I'd gone to that much trouble, I might as well write the whole > shebang in Java. In fact, I prefer picking "the right tool for the > job" for each part of a project and Erlang doesn't fit every niche so > sometimes turning to Java, Python and other things is the correct > choice. Erlang does have some decent open source libraries for doing > database access (to postgres at least) and I've seen a very good > closed source Oracle driver written as a port_driver using the OCL > libraries - quick and stable, but sadly not available to the general > public. The author claims he'll help do a rewrite when native > processes get added to Erlang. :) > > I agree with the sentiment. Java will indeed usually get the least friction for any implementation, which for the Enterprise-minded is usually reason enough to use it for everything. However, in many places one could find instances where building at least a prototype in Erlang might be the Right Thing. Several back-end or even front-end services I have built I would certainly have knocked up in Erlang very rapidly, just as an idea to Show The Boss or even as a real deployment candidate. In all these instances, the database was either Oracle or MS-SQL, with reasonably low DB access performance expectations. Now, like you said above, just dropping into another VM for whatever service is going to "look bad". Certainly if you've really built it from scratch for your pet project, but what if we were talking about something that's been battle-proven? That maybe speaks Enterprise lingo? Maybe put a nice face on it, for instance - add a RESTful JSON interface or something, as a diversion? :) Like you said, the Enterprise is often willing to take a lot of crap just to have something that looks "standard" - and deploying a Lean JDBC Proxy can go down relatively easily. > > > > I think that what is truly missing is an agreed-upon, standard EDBC API > > definition. IMHO one that is based on JDBC could certainly be good > enough. > > Once that is in place, implementation is just a detail :) > > Erlang Community, time to claim your place in the Enterprise!!!! :) > > > > I kind of agree that a standard API would be good. Unless it comes > from Ericsson or gets into the OTP distro, it's not really going to be > a standard though. If someone starts one, they really need to get on > this list and push people for feedback. I'd also suggest making it as > thin a veneer as possible on top of the real library/application being > used, as this is less likely to introduce problems. > > Right. Something that I'd use the JDBC Proxy Abomination suggested above is a fallback. In that case, you're not expecting blazing performance - however, you'll have a stable backend to bang your API out on. Gradually adding clean Erlang backends can come later. > I think a big part of the reason why medium/large enterprises don't > tend to use Erlang as a core development language, actually has > nothing to do with technology. It's a resourcing issue. It's harder to > find real Erlang expertise (i.e., commercial experience) than it is to > find Java and/or .NET developers and that increases both cost and > risk. I certainly think that Erlang's productivity and other factors > (like its built-in support for making fault tolerant applications) can > actually outweigh the initial cost of hiring good developers, but > again this is a risk that many big businesses (which are mainly run by > accountants in my experience) won't take. Start-ups and niche sectors > are a completely different kettle of fish. > > I think that's spot on. Neither should they change this approach. If you want to leap forward in language/tool technology in the Enterprise, there's always stuff like Scala (which is an excellent step up from Java, and starting to get some Traction). > Finally, I agree that JDBC has some good patterns to follow, but it's > whole design is very Object Oriented (i.e., mutable state oriented) > and therefore not much of it is really a good fit for Erlang. Things > that might translate nicely as features are (IMHO): > > 1. Runtime discovery of drivers > 2. Runtime discovery of data source(s) a la JNDI - this is very useful > in Java applications and something like it (maybe gproc based?) would > be nice > 3. Standard API for dealing with connections (and possibly connection > pools, although I'm not sure I'd want to open that particular can of > worms myself) > 4. Standard API for working with database metadata (e.g., > INFORMATION_SCHEMA, system views and the like) > 5. Standard API for working with result sets (e.g., moving/scrolling, > indexed access to columns in the current row, etc) > 6. QLC queries for interacting with result sets? > > Not sure if that last one is really a good idea - it's very late here > and I've had a busy week. > > AFAICT good API design for Erlang doesn't usually involve exposing > internal data structures (records, etc) to the end user and any state > is usually hidden as much as possible. My assumption is that the > various API modules involved would usually just return an opaque > handle to the user, which is then used during further calls. I > certainly wouldn't go off using parameterised modules as they're not > supported. So you'd get something more like: > > {ok, DSHandle} = edbc_datasource:get_named_datasource(postgres_test), > {ok, ConnHandle} = edbc_connection:open(DSHandle, [PerConnOptions]), > {ok, TxnHandle} = edbc_transaction:begin(ConnHandle, [{isolation, > read_committed]), > try > %% fall back to normal (not prepared) statement when this doesn't > work... > {ok, StatementHandle} = ebdc_connection:prepare_statement(ConnHandle, > SQL), > {ok, StatementHandle} = edbc_statement:bind_named_params([{id, > 12345}, {customer_name, "Mr Smith"}]), > > {ok, Cursor} = edbc_connection:execute_query(ConnHandle, > StatementHandle), > edbc_rowset:foreach(fun(Row, Col, Value) -> io:format("~p:~p = > ~p~n", [Row, Col, Value]) end, Cursor), > > %% version of execute/2 that takes raw SQL instead of prepared > statements > edbc_connection:execute_update(ConnHandle, "INSERT INTO Food > VALUES ('Ice Cream', 'Vanilla')"), > {ok, committed} = edbc_transaction:commit(TxnHandle) > catch > _:_ -> {ok, aborted} = edbc_transaction:rollback(TxnHandle) > after > {ok, closed} = edbc_connection:close(ConnHandle) > > > Probably a lot of these calls to specific modules could be hidden > behind a slightly higher level API: > > {ok, Conn} = edbc:open_connection(postgres_test), > Results = edbc:map(Conn, fun do_with_each_cell/2, SqlQuery), > %% or > Tree = edbc:fold(Conn, Query, > fun(Row, Col, Val, Acc) -> gb_trees:enter({Row, Col}, Val, Acc) end), > etc..... > > You're right that having a good standard API against which all of the > various implementations could be used is a very nice idea. The > question is, how close are the existing libraries/applications to a > *nice* API (whatever that is) and how much work is involved in > bridging between them. This is why I'd like to see if the binding > between API and implementation can be generated - I'd like to avoid > having state in the API and let the underlying implementations deal > with their own states, servers, errors, etc. > AFAICT, the existing APIs are similar to a severely cut-down JDBC API. So nothing very surprising there - I'm guessing most (if not all) have been built on a per-need basis. I agree that one should try to veer as much away from heavily-stateful APIs, and this can probably be avoided. However, since we ARE dealing with RDBMS's, some of the flavour of using them is bound to come through the API - connection pooling, transactions and error handling/reporting should definitely be part of any API that expects to be widely useful. I'd also add an Erlang-y flavour to some operations, like streaming of result sets, to work with the platform's grain and not so much against it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Jul 31 21:10:45 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 31 Jul 2011 20:10:45 +0100 Subject: [erlang-questions] trouble with erlang or erlang is a ghetto In-Reply-To: References: <855F2E6C-06DB-4B15-99A2-6B06DCD6EBB4@gmail.com> <4E2F2C01.3060801@dev-extend.eu> <125826B8-100A-43DB-A40F-46543FFC8FD8@cs.otago.ac.nz> <4E2FAB4F.7000903@gmail.com> <75460EFA-A6EE-4001-9794-5437E1FF705A@cs.otago.ac.nz> Message-ID: > > I agree with the sentiment. Java will indeed usually get the least friction > for any implementation, which for the Enterprise-minded is usually reason > enough to use it for everything. > However, in many places one could find instances where building at least a > prototype in Erlang might be the Right Thing. Several back-end or even > front-end services I have built I would certainly have knocked up in Erlang > very rapidly, just as an idea to Show The Boss or even as a real deployment > candidate. In all these instances, the database was either Oracle or MS-SQL, > with reasonably low DB access performance expectations. > Now, like you said above, just dropping into another VM for whatever service > is going to "look bad". Certainly if you've really built it from scratch for > your pet project, but what if we were talking about something that's been > battle-proven? That maybe speaks Enterprise lingo? Maybe put a nice face on > it, for instance - add a RESTful JSON interface or something, as a > diversion? :) Like you said, the Enterprise is often willing to take a lot > of crap just to have something that looks "standard" - and deploying a Lean > JDBC Proxy can go down relatively easily. > I've written a few real production systems in Erlang, but they were all OSS applications and it was the right fit. Even in a telco, most BSS applications tend not to be written in Erlang. I'm not saying that's right or good - I personally would rather build something in Erlang than Java - but there we have it. > > Right. > Something that I'd use the JDBC Proxy Abomination suggested above is a > fallback. In that case, you're not expecting blazing performance - however, > you'll have a stable backend to bang your API out on. Gradually adding clean > Erlang backends can come later. > Yeah I get that it will work ok, it just feels a bit weird not to use existing native implementations that are known to work. ESL have a postgres driver which I'm assuming (?) is used in production applications. The database support for Zotonic is also very solid in my experience. As I said, a guy at work wrote a driver based Oracle back end and it's absolutely rock solid in production - the app quite data access centric and has never once failed in the last 3 years whilst it deals with a reasonable load (avg. few thousand regular/daily users with peak times dipping into five figures). > > I think that's spot on. > Neither should they change this approach. > If you want to leap forward in language/tool technology in the Enterprise, > there's always stuff like Scala (which is an excellent step up from Java, > and starting to get some Traction). > Like I said, we use Erlang at work where it's deemed appropriate. Nobody there thinks Erlang "is a ghetto" but they do see it as niche and the fact that they pay sub contractors a lot more for it than bog standard Java applications is something they think about a lot. Having said that IMHO the fact that these Erlang based systems never seem to go wrong is testament to what I call the "you get what you pay for" effect combined with "some tools are better than others". Just my opinion though. > > AFAICT, the existing APIs are similar to a severely cut-down JDBC API. So > nothing very surprising there - I'm guessing most (if not all) have been > built on a per-need basis. That's why I'd like to start by fronting a well established API - like the ESL postgres driver - just to see what's already there and build on it in the general case. > I agree that one should try to veer as much away from heavily-stateful APIs, > and this can probably be avoided. Yes - just avoiding exposing internal data structures by providing a handle is probably enough to do this. The various modules can actually return whatever (real state if required) but the API should just hide this by exporting opaque types only: -opaque connection_handle :: term(). -export_type([connection_handle/0]). > However, since we ARE dealing with RDBMS's, some of the flavour of using > them is bound to come through the API - connection pooling, transactions and > error handling/reporting should definitely be part of any API that expects > to be widely useful. I'd also add an Erlang-y flavour to some operations, > like streaming of result sets, to work with the platform's grain and not so > much against it. > Yes all these things (pooling, transactions, etc) are necessary. I maintain that you can provide them to the user without the implementation details leaking though. I also agree with the notion of having operations that are Erlang-y. Not sure what the best thing to do with streaming result sets would be, but I suspect looking at the existing implementations would clarify what people expect to see. For sure you don't always want to have to wait until the entire result set has been processed before you can do anything. I do think having operations that convert result sets using some transformation fun/function would be useful though - you might think of there as the Erlang equivalent of the Spring Framework JdbcTemplate ResultSetConverter or RowMapper interfaces. Except in Erlag you just a function, plain and simple. Is your JDBC thing open source? I might have a stab at this just for fun - maybe fronting your library and the ESL postgres API as the first cut. Cheers, Tim