From colm.dougan@REDACTED Sat May 1 00:17:43 2010 From: colm.dougan@REDACTED (Colm Dougan) Date: Fri, 30 Apr 2010 15:17:43 -0700 Subject: [erlang-questions] page counter - thx for everyones help In-Reply-To: References: Message-ID: Wes, It would be typical to move the "extract page count" code into a function. Depending on how happy you are depend on the format of the page you are scraping you could use something like this (see below). Note: I've used the default "http" library here rather than "ibrowse" as I don't have ibrowse installed. If you decide to use "re" instead then you only have to change the extract_page_count function. Typically lists:nth isn't very erlangy. It would probably be more typical (and more tolerant to the page changing) to iterate over Lines looking for a given pattern. Colm -define(MAGIC_LINE_NUMBER, 424). -define(URL, ""http://192.168.1.1/printer/configu.html?autoref=0&weblang=0"). getc() -> inets:start(), {ok, {_StatusLine, _Headers, Body}} = http:request(?URL), Lines = string:tokens(Body, "\r\n"), PageCount = extract_page_count(lists:nth(?MAGIC_LINE_NUMBER, Lines)), io:format("COUNT [~p]~n", [PageCount]), ok. extract_page_count("Page Counter" ++ Rest) -> {Count, _} = string:to_integer(Rest), Count. On Fri, Apr 30, 2010 at 12:17 PM, Wes James wrote: > Here is the code so far: > > -module('gpc'). > > -export([start/0, start/1, getc/0]). > > > start() -> > ? ? ? ?ibrowse:start(), > ? ? ? ?timer:apply_interval(3000, gpc, getc, []). > > start(Interval) -> > ? ? ? ?ibrowse:start(), > ? ? ? ?timer:apply_interval(Interval*1000, gpc, getc, []). > > getc() -> > ? ? ? ?S = string:substr( > ? ? ? ? ? ? ? ?binary_to_list( > ? ? ? ? ? ? ? ? ? ? ? ?lists:nth( > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?424, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?re:replace( > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?erlang:element( > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?4, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ibrowse:send_req("http://192.168.1.1/printer/configu.html?autoref=0&weblang=0", > [], get) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"\\n", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[global] > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?) > ? ? ? ? ? ? ? ? ? ? ? ?) > ? ? ? ? ? ? ? ?), > ? ? ? ?34), > ? ? ? ?S2=list_to_integer(string:substr(S, 1, string:len(S)-10)), > ? ? ? ?io:format("~w~n", [S2]). > > > So this by default gives me a page count every 3 seconds or I can put > in gpc:start(10) for 10 second intervals. > > What I would like to do is take daily page counts and stick them some > where. ?I could make a file on the system with the date as the name > (20100430.txt) and just write the value there or I could write the > value to mnesia and keep track via the data base. ?I would be doing a > current count - yesterday count to get a total count for just each > day. ?I'm doing this so I can keep track of what the students in a > university computer lab are printing each day. > > If you have any suggestions, please drop them in this list :) > > I'll be experimenting anyway since I've yet to read or write to a file > with erlang on a disc or gone beyond just reading what mnesia can do > via web page info. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Sat May 1 00:47:53 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Apr 2010 16:47:53 -0600 Subject: [erlang-questions] page counter - thx for everyones help In-Reply-To: References: Message-ID: On Fri, Apr 30, 2010 at 4:17 PM, Colm Dougan wrote: > Wes, > > It would be typical to move the "extract page count" code into a > function. ?Depending on how happy you are depend on the format of the > page you are scraping you could use something like this (see below). > Note: I've used the default "http" library here rather than "ibrowse" > as I don't have ibrowse installed. ?If you decide to use "re" instead > then you only have to change the extract_page_count function. > > Typically lists:nth isn't very erlangy. ?It would probably be more > typical (and more tolerant to the page changing) to iterate over Lines > looking for a given pattern. > > Colm > > -define(MAGIC_LINE_NUMBER, 424). > -define(URL, ""http://192.168.1.1/printer/configu.html?autoref=0&weblang=0"). > > getc() -> > ? ? ? ?inets:start(), > ? ? ? ?{ok, {_StatusLine, _Headers, Body}} = http:request(?URL), > ? ? ? ?Lines = string:tokens(Body, "\r\n"), > ? ? ? ?PageCount = extract_page_count(lists:nth(?MAGIC_LINE_NUMBER, Lines)), > ? ? ? ?io:format("COUNT [~p]~n", [PageCount]), > ? ? ? ?ok. > > extract_page_count("Page Counter" ++ Rest) -> > ? ? ? ?{Count, _} = string:to_integer(Rest), > ? ? ? ?Count. Colm, Thx! I like this a lot better without having to use ibrowse. I appreciate this info! -wes From dmurray@REDACTED Sat May 1 01:12:00 2010 From: dmurray@REDACTED (David N Murray) Date: Fri, 30 Apr 2010 19:12:00 -0400 (EDT) Subject: [OT] Re: [erlang-questions] Re: Autoreply: Re: [erlang-questions] calling timer_in In-Reply-To: References: <15fadd01cae895$87b92770$0165010a@mail2world.com> Message-ID: On Apr 30, Wes James scribed: > > Every time I send a message to the list today, I get this persons > autoreply. What happens on this list when this happens - list manager > boots them or ignore? I've been getting them to. He has a borked email client. It should not respond to "Precedence: Bulk" in the email header, which is the norm for lists. Dave From per.melin@REDACTED Sat May 1 01:35:55 2010 From: per.melin@REDACTED (Per Melin) Date: Sat, 1 May 2010 01:35:55 +0200 Subject: [erlang-questions] Re: http or httpc? In-Reply-To: References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> Message-ID: On Thu, Apr 29, 2010 at 1:55 PM, zabrane Mikael wrote: > I'll give httpc a try. Be warned that in R13B4 there seems to be a potentially serious bug in httpc. For each request you make the ETS table httpc_manager__handler_db will grow with one entry, indefinitely. With enough requests over time that can of course consume all memory and crash the node. 1> inets:start(). ok 2> [ http:request("http://127.0.0.1/") || _ <- lists:seq(1,10000) ], ok. ok 3> ets:info(httpc_manager__handler_db, size). 10000 And R13B4 is not the first OTP version where the http client has serious issues. I've learned to stay away from it when I can. From mabrek@REDACTED Sat May 1 09:44:49 2010 From: mabrek@REDACTED (mabrek) Date: Sat, 1 May 2010 11:44:49 +0400 Subject: is EEP 31 implemented in R13B04 ? Message-ID: Hello, http://www.erlang.org/eeps/eep-0031.html#reference-implementation states that it will be implemented in R13B04, but I can't find it in my distribution (ubuntu packages from ppa https://edge.launchpad.net/~erlang-dev/+archive/ppa ) Is it implemented in R13B04 or how else can I find the implementation? Best regards, Anton Lebedevich. From zabrane3@REDACTED Sat May 1 10:04:57 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Sat, 1 May 2010 10:04:57 +0200 Subject: [erlang-questions] Re: http or httpc? In-Reply-To: References: <89298b8d-5ecd-4fde-9a21-d1b23582d140@g21g2000yqk.googlegroups.com> Message-ID: Exactly what I noticed Per. ibrowse will doi the job for me now. Thanks again for the hints guys ... 2010/5/1 Per Melin > On Thu, Apr 29, 2010 at 1:55 PM, zabrane Mikael > wrote: > > I'll give httpc a try. > > Be warned that in R13B4 there seems to be a potentially serious bug in > httpc. > > For each request you make the ETS table httpc_manager__handler_db will > grow with one entry, indefinitely. With enough requests over time that > can of course consume all memory and crash the node. > > 1> inets:start(). > ok > 2> [ http:request("http://127.0.0.1/") || _ <- lists:seq(1,10000) ], ok. > ok > 3> ets:info(httpc_manager__handler_db, size). > 10000 > > And R13B4 is not the first OTP version where the http client has > serious issues. I've learned to stay away from it when I can. > From erlang@REDACTED Sat May 1 12:03:27 2010 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 1 May 2010 12:03:27 +0200 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: I think I'd have pretended that the input string were Erlang tokens. I'd then use the Erlang tokeniser and pattern match the output, thusly: 1 > X="Page Counter4880". .. 2 > hd([I || {integer,_,I} <- element(2, erl_scan:string(X))]). 4880 erl_scan:string is your friend :-) /Joe [I know it's cheating, so your don't have to tell me] On Fri, Apr 30, 2010 at 9:49 PM, Wes James wrote: > On Fri, Apr 30, 2010 at 1:46 PM, Colm Dougan wrote: >> If you already found the offset of the start of the number then you >> can use string:to_inetger (or more generally io_lib:fread) to extract >> the number, regardless of it being variable length. >> >> Eshell V5.7.5 ?(abort with ^G) >> 1> string:to_integer("4880"). >> {4880,""} >> >> Eshell V5.7.5 ?(abort with ^G) >> 1> io_lib:fread("~d", "4880"). >> {ok,[4880],[]} >> >> Colm > > > Hey that's cool! ?Thx Colm! > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuncer.ayaz@REDACTED Sat May 1 12:53:14 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 1 May 2010 12:53:14 +0200 Subject: [erlang-questions] is EEP 31 implemented in R13B04 ? In-Reply-To: References: Message-ID: On Sat, May 1, 2010 at 9:44 AM, mabrek wrote: > Hello, > > http://www.erlang.org/eeps/eep-0031.html#reference-implementation > states that it will be implemented in R13B04, but I can't find it in > my distribution (ubuntu packages from ppa > https://edge.launchpad.net/~erlang-dev/+archive/ppa ) > Is it implemented in R13B04 or how else can I find the implementation? No it isn't. It was planned for R13B04. Right now it's planned for R14A/R14B. See Kenneth Lundin's slides from Erlang Factory SF Bay 2010. From hynek@REDACTED Sat May 1 15:01:09 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 1 May 2010 15:01:09 +0200 Subject: [erlang-questions] FYI, Erlang reference in conference about the economy In-Reply-To: References: Message-ID: Nice, but I disappoint you. It has nothing to do with erlang language [1] but with this guy [2] and his formula. So Frank Kelly thinks that not our lovely language but applied mathematics saves the world. [1] http://en.wikipedia.org/wiki/Erlang_(programming_language) [2] http://en.wikipedia.org/wiki/Agner_Krarup_Erlang On Fri, Apr 30, 2010 at 9:54 AM, Mats wrote: > This conference[1] at the University of Cambridge "Network and Systemic > Risks" is using Erlang to explain how we may save the world.. > > I love this quote[2]: > "..the telephony network is arguably at the moment the largest machine that > the human race has constructed, arguably because the Internet is bigger now > with the same issues.." > > What do you think, interesting? :) > > Mats Westin > > > [1] http://www.youtube.com/watch?v=bz6z3BEI-1E#t=3m45s > [2] http://www.youtube.com/watch?v=bz6z3BEI-1E#t=6m10s > > -> http://se.linkedin.com/in/matswestin > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From comptekki@REDACTED Sat May 1 15:03:35 2010 From: comptekki@REDACTED (Wes James) Date: Sat, 1 May 2010 07:03:35 -0600 Subject: inets and https Message-ID: I can find many things about http:request but I can't find much on getting an https page or posting to an https page. I would like to also grab a cookie (on the version page grab) and then carry the cookie along with the future posts to an https site. Does anyone have any examples of this? I couldn't find one thing via google about even getting an https page and I just don't understand http://www.erlang.org/doc/apps/ssl/index.html docs. thx, -wes From hd2010@REDACTED Sat May 1 15:30:28 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 01 May 2010 15:30:28 +0200 Subject: terminal codes Message-ID: <4BDC2CF4.8030501@eonblast.com> Can I print terminal color codes out of Erlang? E.g. \e[30m? The io: functions seem to force control codes to be printable. Thanks! Henning From steven.charles.davis@REDACTED Sat May 1 15:44:45 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 1 May 2010 06:44:45 -0700 (PDT) Subject: terminal codes In-Reply-To: <4BDC2CF4.8030501@eonblast.com> References: <4BDC2CF4.8030501@eonblast.com> Message-ID: Not sure if this is the answer you are looking for, but you can "print" control codes... io:format(<<"\0">>, []). /s On May 1, 8:30?am, Henning Diedrich wrote: > Can I print terminal color codes out of Erlang? > > E.g. \e[30m? > > The io: functions seem to force control codes to be printable. > > Thanks! > Henning > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From rapsey@REDACTED Sat May 1 15:56:34 2010 From: rapsey@REDACTED (Rapsey) Date: Sat, 1 May 2010 15:56:34 +0200 Subject: [erlang-questions] inets and https In-Reply-To: References: Message-ID: Not sure about http module, but ibrowse supports https. You only need to add [{is_ssl, true},{ssl_options,[]}] to the options of the request. Sergej On Sat, May 1, 2010 at 3:03 PM, Wes James wrote: > I can find many things about http:request but I can't find much on > getting an https page or posting to an https page. I would like to > also grab a cookie (on the version page grab) and then carry the > cookie along with the future posts to an https site. Does anyone have > any examples of this? I couldn't find one thing via google about even > getting an https page and I just don't understand > http://www.erlang.org/doc/apps/ssl/index.html docs. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Sat May 1 15:59:09 2010 From: comptekki@REDACTED (Wes James) Date: Sat, 1 May 2010 07:59:09 -0600 Subject: [erlang-questions] parsing text In-Reply-To: References: Message-ID: On Sat, May 1, 2010 at 4:03 AM, Joe Armstrong wrote: > I think I'd have pretended that the input string were Erlang tokens. > I'd then use the Erlang tokeniser and pattern match the output, thusly: > > 1 > X="Page Counter4880". > .. > 2 > hd([I || {integer,_,I} <- element(2, erl_scan:string(X))]). > 4880 > > erl_scan:string is your friend :-) > > /Joe Thx. Joe. I'm a sys admin at a university and everyone's help here will make it easier for me learn how to parse the input of a brother printer config page that has the page count. Yesterday I went through some mnesia tutorials about creating tables and putting data in. I plan on using erlang:date() to put the count in a page_count record. thx, -wes From comptekki@REDACTED Sat May 1 16:02:25 2010 From: comptekki@REDACTED (Wes James) Date: Sat, 1 May 2010 08:02:25 -0600 Subject: [erlang-questions] inets and https In-Reply-To: References: Message-ID: On Sat, May 1, 2010 at 7:56 AM, Rapsey wrote: > Not sure about http module, but ibrowse supports https. You only need to add > [{is_ssl, true},{ssl_options,[]}] to the options of the request. > > > > Sergej, Thx. Is there a way to get the output in the format like http:request does (list of lines)? With ibrowse (which I've tried during the week), it sends back everything, even \n in the code that have to be removed for some decent parsing, it seemed. thx, wes From g9414002.pccu.edu.tw@REDACTED Sat May 1 16:19:18 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Sat, 1 May 2010 22:19:18 +0800 Subject: [erlang-questions] terminal codes In-Reply-To: <4BDC2CF4.8030501@eonblast.com> References: <4BDC2CF4.8030501@eonblast.com> Message-ID: I made a Erlang script and got output that seems correct. #!/usr/bin/env escript -compile(export_all). main([]) -> io:format("\e[1;36mabc\e[0;37m"). I checked it by pasting what printed into a telnet BBS and viewing it effect. It works. So the problem may shift to how to make Erlang shell support terminal color codes. On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich wrote: > Can I print terminal color codes out of Erlang? > > E.g. \e[30m? > > The io: functions seem to force control codes to be printable. > > Thanks! > Henning > > From erlangy@REDACTED Sat May 1 16:52:01 2010 From: erlangy@REDACTED (Michael McDaniel) Date: Sat, 1 May 2010 07:52:01 -0700 Subject: [erlang-questions] inets and https In-Reply-To: References: Message-ID: <20100501145200.GH5061@delora.autosys.us> On Sat, May 01, 2010 at 07:03:35AM -0600, Wes James wrote: > I can find many things about http:request but I can't find much on > getting an https page or posting to an https page. I would like to > also grab a cookie (on the version page grab) and then carry the > cookie along with the future posts to an https site. Does anyone have > any examples of this? I couldn't find one thing via google about even > getting an https page and I just don't understand > http://www.erlang.org/doc/apps/ssl/index.html docs. > > thx, > > -wes > ________________________________________________________________ I suggest installing the provided Erlang documentation locally on your workstation. Then, you can search locally. For instance, in my browser I went to file:///usr/local/lib/erlang/lib/inets-5.2/doc/html/http.html and searched on ssl. First find was in a big green Note box ... "If the scheme https is used the ssl application needs to be started." So, I did the following ... $ erl Erlang R13B03 (erts-5.7.4) [smp:1:1] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> inets:start(). ok 2> ssl:start(). ok 3> http:request("https://some_webserver.com"). which worked fine. ~Michael From hd2010@REDACTED Sat May 1 19:28:59 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 01 May 2010 19:28:59 +0200 Subject: [erlang-questions] terminal codes In-Reply-To: References: <4BDC2CF4.8030501@eonblast.com> Message-ID: <4BDC64DB.1040700@eonblast.com> Yes! Thanks. Is there a way to persuade the Erlang shell to let terminal color codes through? Thanks, Henning ??? (Yau-Hsien Huang) wrote: > I made a Erlang script and got output that seems correct. > > #!/usr/bin/env escript > -compile(export_all). > main([]) -> > io:format("\e[1;36mabc\e[0;37m"). > > I checked it by pasting what printed into a telnet BBS and viewing it > effect. It works. > So the problem may shift to how to make Erlang shell support terminal color > codes. > > On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich wrote: > > >> Can I print terminal color codes out of Erlang? >> >> E.g. \e[30m? >> >> The io: functions seem to force control codes to be printable. >> >> Thanks! >> Henning >> >> >> > > From hd2010@REDACTED Sat May 1 19:32:40 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 01 May 2010 19:32:40 +0200 Subject: [erlang-questions] terminal codes In-Reply-To: References: <4BDC2CF4.8030501@eonblast.com> Message-ID: <4BDC65B8.7000707@eonblast.com> This way of involing a program seems to prevent it as well. erl -s test run -s init stop What alternate way would allow control codes to get propagated unchanched? Thanks! Henning ??? (Yau-Hsien Huang) wrote: > I made a Erlang script and got output that seems correct. > > #!/usr/bin/env escript > -compile(export_all). > main([]) -> > io:format("\e[1;36mabc\e[0;37m"). > > I checked it by pasting what printed into a telnet BBS and viewing it > effect. It works. > So the problem may shift to how to make Erlang shell support terminal color > codes. > > On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich wrote: > > >> Can I print terminal color codes out of Erlang? >> >> E.g. \e[30m? >> >> The io: functions seem to force control codes to be printable. >> >> Thanks! >> Henning >> >> >> > > From erlang@REDACTED Sat May 1 20:00:26 2010 From: erlang@REDACTED (Stefan Marr) Date: Sat, 1 May 2010 20:00:26 +0200 Subject: [erlang-questions] terminal codes In-Reply-To: <4BDC65B8.7000707@eonblast.com> References: <4BDC2CF4.8030501@eonblast.com> <4BDC65B8.7000707@eonblast.com> Message-ID: Hi Henning: I have implemented a small snake game in Erlang using ANSI terminal control sequences, and it is started as follows: #!/bin/sh stty -echo -echok -icanon min 1 time 0 erl -pa ./ -run main start -run init stop -noshell stty echo echok icanon If I remember correctly, the stty is not relevant for you, it should set the shell to send all characters to Erlang, without awaiting a new-line. Hope that helps. Best regards Stefan On 01 May 2010, at 19:32, Henning Diedrich wrote: > This way of involing a program seems to prevent it as well. > > erl -s test run -s init stop > > What alternate way would allow control codes to get propagated unchanched? > > Thanks! > Henning > > > ??? (Yau-Hsien Huang) wrote: >> I made a Erlang script and got output that seems correct. >> >> #!/usr/bin/env escript >> -compile(export_all). >> main([]) -> >> io:format("\e[1;36mabc\e[0;37m"). >> >> I checked it by pasting what printed into a telnet BBS and viewing it >> effect. It works. >> So the problem may shift to how to make Erlang shell support terminal color >> codes. >> >> On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich wrote: >> >> >>> Can I print terminal color codes out of Erlang? >>> >>> E.g. \e[30m? >>> >>> The io: functions seem to force control codes to be printable. >>> >>> Thanks! >>> Henning >>> >>> >>> >> >> -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 From kiszl@REDACTED Sat May 1 20:38:10 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 01 May 2010 20:38:10 +0200 Subject: closure on function argument Message-ID: <4BDC7512.60604@tmit.bme.hu> Hi, If I have /F = fun(Y) -> fun(Y) -> ok end end/, and /G = F('z')/, I would expect /G/ to be /fun('z') -> ok end/. Apparently, it is /fun(Y) -> ok end/, as calling it with any argument returns /ok/ and never causes a function clause exception. I guess that the inner /Y/ being an argument "automatically" shadows the outer one. A workaround I found is using guards: /F = fun(Y) -> fun(X) when X == Y -> ok end end/. The question is: is there a way to create a direct binding between the inner and outer /Y/'s in the original example? Thank you, Zoltan. From mononcqc@REDACTED Sat May 1 21:08:17 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 1 May 2010 15:08:17 -0400 Subject: [erlang-questions] closure on function argument In-Reply-To: <4BDC7512.60604@tmit.bme.hu> References: <4BDC7512.60604@tmit.bme.hu> Message-ID: A closure is nothing but a regular function with its own independent scope. Sorry if the following lines are too basic, I don't know how much you know. If you already know all the rules about scope, then just skip down to the end of my post. Imagine each function as a dictionary of variables. base() -> A = 1, B = 3, C = 42, F = fun(F) -> E = 17 D = E + B end. In base/0, the dictionary contains 3 entries: A, B, C and F. A contains 1, B contains 3, C contains 42 and F contains the closure/anonymous function. This anonymous function has its own independent scope. In its scope, there's the variable E, which is worth 17. Then there's D, which is worth the result of C + B and finally there's F, which is passed as a parameter. Here, the variable B has no value in the closure's scope. In this case, it goes to the parent level. B is worth 3 because the closure was defined within base/0, and so it can borrow definitions from there. D thus has the value 17+3, which is 20. That's because the closure inherits the parent's scope. This means that had the anonymous function been declared the following way: F = fun(F) -> E = 17 D = E + B, A = 10 end. there would have been an error. Here, 'A' isn't defined in the parent's scope, which has been inherited by the closure. As such, reassigning it the value '10' breaks the rules of single assignment. So what about F? F is defined as an argument in the closure. Then it will take whatever argument the closure takes. It doesn't need to go see in base/0's scope. In fact, it overwrites that entry. This is why you will get compiler warnings on such lines. ------ SKIP HERE ------- To answer your question directly, the theory above should explain why 'F = fun(Y) -> fun(Y) -> ok end end' doesn't give the expected results The second anonymous function redefines the 'Y' that was in the parent's scope. There will be no matching possible and doing it through guards really is the only way I know of to do such pattern matching. If you do want an exception, you might prefer the form fun(A) -> fun(B) -> A = B end end, which will return the right value if they match and die otherwise. They're pretty much the same though. On Sat, May 1, 2010 at 2:38 PM, Zoltan Lajos Kis wrote: > Hi, > > If I have /F = fun(Y) -> fun(Y) -> ok end end/, and /G = F('z')/, I would > expect /G/ to be /fun('z') -> ok end/. > Apparently, it is /fun(Y) -> ok end/, as calling it with any argument > returns /ok/ and never causes a function clause exception. > I guess that the inner /Y/ being an argument "automatically" shadows the > outer one. > > A workaround I found is using guards: /F = fun(Y) -> fun(X) when X == Y -> > ok end end/. > The question is: is there a way to create a direct binding between the > inner and outer /Y/'s in the original example? > > Thank you, > Zoltan. > > > From rvirding@REDACTED Sat May 1 22:56:44 2010 From: rvirding@REDACTED (Robert Virding) Date: Sat, 1 May 2010 22:56:44 +0200 Subject: [erlang-questions] closure on function argument In-Reply-To: <4BDC7512.60604@tmit.bme.hu> References: <4BDC7512.60604@tmit.bme.hu> Message-ID: The variables in the arguments of a fun, Y in your example, are always "fresh" variables and never imported from the funs environment. If you wanted to generate a fun which matches against variables imported from its environment then you need to put an explicit test in the guard. In your case: F = fun(Y) -> fun(Y1) when Y1 == Y -> ok end end. Then you get 8> F = fun(Y) -> fun(Y1) when Y1 == Y -> ok end end. #Fun 9> G = F(z). #Fun 10> G(z). ok 11> G(a). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(a) 12> Robert On 1 May 2010 20:38, Zoltan Lajos Kis wrote: > Hi, > > If I have /F = fun(Y) -> fun(Y) -> ok end end/, and /G = F('z')/, I would > expect /G/ to be /fun('z') -> ok end/. > Apparently, it is /fun(Y) -> ok end/, as calling it with any argument > returns /ok/ and never causes a function clause exception. > I guess that the inner /Y/ being an argument "automatically" shadows the > outer one. > > A workaround I found is using guards: /F = fun(Y) -> fun(X) when X == Y -> > ok end end/. > The question is: is there a way to create a direct binding between the inner > and outer /Y/'s in the original example? > > Thank you, > Zoltan. > > > From hd2010@REDACTED Sat May 1 23:09:13 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 01 May 2010 23:09:13 +0200 Subject: [erlang-questions] terminal codes In-Reply-To: References: <4BDC2CF4.8030501@eonblast.com> <4BDC65B8.7000707@eonblast.com> Message-ID: <4BDC9879.20807@eonblast.com> -noshell does the trick! erl -s testtest run -s init stop -noshell Kind of logical in hindsight ... Beautiful, thanks! Can I see the snake game? Henning Stefan Marr wrote: > Hi Henning: > > I have implemented a small snake game in Erlang using ANSI terminal control sequences, and it is started as follows: > > #!/bin/sh > stty -echo -echok -icanon min 1 time 0 > erl -pa ./ -run main start -run init stop -noshell > stty echo echok icanon > > > If I remember correctly, the stty is not relevant for you, it should set the shell to send all characters to Erlang, without awaiting a new-line. > > Hope that helps. > > Best regards > Stefan > > On 01 May 2010, at 19:32, Henning Diedrich wrote: > > >> This way of involing a program seems to prevent it as well. >> >> erl -s test run -s init stop >> >> What alternate way would allow control codes to get propagated unchanched? >> >> Thanks! >> Henning >> >> >> ??? (Yau-Hsien Huang) wrote: >> >>> I made a Erlang script and got output that seems correct. >>> >>> #!/usr/bin/env escript >>> -compile(export_all). >>> main([]) -> >>> io:format("\e[1;36mabc\e[0;37m"). >>> >>> I checked it by pasting what printed into a telnet BBS and viewing it >>> effect. It works. >>> So the problem may shift to how to make Erlang shell support terminal color >>> codes. >>> >>> On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich wrote: >>> >>> >>> >>>> Can I print terminal color codes out of Erlang? >>>> >>>> E.g. \e[30m? >>>> >>>> The io: functions seem to force control codes to be printable. >>>> >>>> Thanks! >>>> Henning >>>> >>>> >>>> >>>> >>> >>> > > > > From hd2010@REDACTED Sat May 1 23:13:41 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 01 May 2010 23:13:41 +0200 Subject: Autoreply In-Reply-To: <16725301cae972$dd2c1e00$0165010a@mail2world.com> References: <16725301cae972$dd2c1e00$0165010a@mail2world.com> Message-ID: <4BDC9985.3080602@eonblast.com> I am so happy for Anders. He found a way to really rub it in. Anders Leven wrote: > Hi! > > I am on vacation, if it is urgent send me an SMS. > > R/ Anders > > _______________________________________________________________ > > _______________________________________________________________ > From steven.charles.davis@REDACTED Sat May 1 23:35:02 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 1 May 2010 14:35:02 -0700 (PDT) Subject: parsing text In-Reply-To: References: Message-ID: Purely out of interest and for offering other approaches, my first instinct was: Bin = <<"Page Counter4880">>, {match, [{Index,Length}]} = re:run(Bin, <<"[0-9]+">>), <<_:Index/binary, Num:Length/binary, _/binary>> = Bin, list_to_integer(binary_to_list(Num)). /s From boris.okner@REDACTED Sun May 2 05:03:21 2010 From: boris.okner@REDACTED (bokner) Date: Sat, 1 May 2010 20:03:21 -0700 (PDT) Subject: Function to syntax tree Message-ID: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> Hi, erl_scan and erl_parse can turn function represented by string into AST. For example, 1> {ok, S, _} = erl_scan:string("fun(X) -> X+1 end."), 1> {ok, P} = erl_parse:parse_exprs(S), 1> [F] = P, erl_syntax:type(F). fun_expr My question is: how do I do the same, but with the function itself instead string representation? I.e., F = fun(X) -> X +1 end, AST = , fun_expr = erl_syntax:type(AST). Thanks, Boris From luismarianoguerra@REDACTED Sun May 2 06:39:18 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sun, 2 May 2010 01:39:18 -0300 Subject: [ANN] efene 0.5 - two languages for the erlang VM - released Message-ID: new release of efene. efene is a programming language that runs on the erlang virtual machine. the idea is to provide an alternative syntax to erlang that is most suitable for people coming from languages like Java, C, C++, C#, Javascript and alternatively python and ruby using ifene (indented efene). the language is almost 100% compatible with erlang (and will be), the compiler allows to translate an efene source file into a readable erlang one or compile it directly to bytecode. It also adds some syntactic sugar in some places to make some tasks easier. changes * complete rewrite of the compiler, less code and more clear, less transformations to get the AST * removed the dependency on the go compiler, now the frontend logic is split between fn.erl and a small shell script. That means: only erlang required to build and use efene. * added a new language called ifene (indented efene), it has the same syntax than efene but the blocks are indented with spaces instead of curly brackets. The compiler will compile depending on the extension (.ifn for ifene .fn for efene) and transform ifene to efene internally before compiling. Tabs are not allowed for indentation in ifene, this is to avoid mixing spaces and tabs. * support for newlines before and after curly brackets and after commas. for example: before this was the only way if A { BodyA } else if B { BodyB } else { BodyElse } now this two forms are also valid if A { BodyA } else if B { BodyB } else { BodyElse } # or if A { BodyA } else if B { BodyB } else { BodyElse } * now commas are required in lists, tuples and function arguments * the try block is now try/catch/after to make it equal to erlang * the receive block is now receive/else receive/after * operator precedence made the same as erlang * tests included to check that efene generates the same ast as erlang, tests cover almost all the language * now records can be declared in efene * bit syntax made the same as erlang for a complete reference see the Language Reference Participate a mailing list is available at librelist just send a mail to efene@REDACTED to subscribe. as first mail you may send a hello world program in efene and present yourself by saying your name, where you are, how did you heard about efene and anything else you would like to say. Resources * Website: http://marianoguerra.com.ar/efene * Blog: http://efene.tumblr.com * Central repo: http://github.com/marianoguerra/efene * Tracker: http://github.com/marianoguerra/efene/issues * Rosetta code page: http://rosettacode.org/wiki/Efene From hd2010@REDACTED Sun May 2 09:52:27 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 09:52:27 +0200 Subject: Debug support on for guards? Message-ID: <4BDD2F3B.2020605@eonblast.com> Hi folks, is there a way to add detail to a function_clause error messages? Be it guards or patterns that don't match? On a crash, I would like to clarify what guard failed, or even warn the programmer that this guard exists when it may not be intuitive. 1 This is written, I believe, as it should: milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro), Mega >= 0, Sec >= 0, Micro >= 0 -> Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). 2 This fails the way I would like it to, giving "negative_input" as reason. But it is written wrong for two (2) counts I believe (I am NOT trying to sneak in a point about ifs): milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro) -> if (Mega >= 0), (Sec >= 0), (Micro >= 0) -> ok; true -> throw(negative_input) end, Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). Thank you very much for taking a look, Henning From hd2010@REDACTED Sun May 2 10:15:00 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 10:15:00 +0200 Subject: [erlang-questions] [ANN] efene 0.5 - two languages for the erlang VM - released In-Reply-To: References: Message-ID: <4BDD3484.6020603@eonblast.com> Very exciting --- can I ask the others what they think is necessary regarding tests to find efene reliable? Thanks, Henning Mariano Guerra wrote: > new release of efene. > > efene is a programming language that runs on the erlang virtual machine. > > the idea is to provide an alternative syntax to erlang that is most > suitable for people coming from languages like Java, C, C++, C#, > Javascript and alternatively python and ruby using ifene (indented > efene). > > the language is almost 100% compatible with erlang (and will be), the > compiler allows to translate an efene source file into a readable > erlang one or compile it directly to bytecode. It also adds some > syntactic sugar in some places to make some tasks easier. > changes > > * complete rewrite of the compiler, less code and more clear, less > transformations to get the AST > * removed the dependency on the go compiler, now the frontend > logic is split between fn.erl and a small shell script. That means: > only erlang required to build and use efene. > * added a new language called ifene (indented efene), it has the > same syntax than efene but the blocks are indented with spaces instead > of curly brackets. The compiler will compile depending on the > extension (.ifn for ifene .fn for efene) and transform ifene to efene > internally before compiling. Tabs are not allowed for indentation in > ifene, this is to avoid mixing spaces and tabs. > * support for newlines before and after curly brackets and after > commas. for example: > > before this was the only way > > > > > if A { > BodyA > } else if B { > BodyB > } else { > BodyElse > } > > > now this two forms are also valid > > > if A { > BodyA > } > else if B { > BodyB > } > else { > BodyElse > } > > # or > > if A > { > BodyA > } > else if B > { > BodyB > } > else > { > BodyElse > } > > * now commas are required in lists, tuples and function arguments > * the try block is now try/catch/after to make it equal to erlang > * the receive block is now receive/else receive/after > * operator precedence made the same as erlang > * tests included to check that efene generates the same ast as > erlang, tests cover almost all the language > * now records can be declared in efene > * bit syntax made the same as erlang > > for a complete reference see the Language Reference > Participate > > a mailing list is available at librelist just send a mail to > efene@REDACTED to subscribe. > > as first mail you may send a hello world program in efene and present > yourself by saying your name, where you are, how did you heard about > efene and anything else you would like to say. > Resources > > * Website: http://marianoguerra.com.ar/efene > * Blog: http://efene.tumblr.com > * Central repo: http://github.com/marianoguerra/efene > * Tracker: http://github.com/marianoguerra/efene/issues > * Rosetta code page: http://rosettacode.org/wiki/Efene > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From hd2010@REDACTED Sun May 2 10:57:08 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 10:57:08 +0200 Subject: [erlang-questions] [ANN] efene 0.5 - two languages for the erlang VM - released In-Reply-To: References: Message-ID: <4BDD3E64.9030907@eonblast.com> Did I overlook that? What syntax are you using for pattern matching, e.g. erl_integer(<>=X), E >= ?LONG_MIN, E =< ?LONG_MAX -> E. Regards, Henning Mariano Guerra wrote: > new release of efene. > > efene is a programming language that runs on the erlang virtual machine. > > the idea is to provide an alternative syntax to erlang that is most > suitable for people coming from languages like Java, C, C++, C#, > Javascript and alternatively python and ruby using ifene (indented > efene). > > the language is almost 100% compatible with erlang (and will be), the > compiler allows to translate an efene source file into a readable > erlang one or compile it directly to bytecode. It also adds some > syntactic sugar in some places to make some tasks easier. > changes > > * complete rewrite of the compiler, less code and more clear, less > transformations to get the AST > * removed the dependency on the go compiler, now the frontend > logic is split between fn.erl and a small shell script. That means: > only erlang required to build and use efene. > * added a new language called ifene (indented efene), it has the > same syntax than efene but the blocks are indented with spaces instead > of curly brackets. The compiler will compile depending on the > extension (.ifn for ifene .fn for efene) and transform ifene to efene > internally before compiling. Tabs are not allowed for indentation in > ifene, this is to avoid mixing spaces and tabs. > * support for newlines before and after curly brackets and after > commas. for example: > > before this was the only way > > > > > if A { > BodyA > } else if B { > BodyB > } else { > BodyElse > } > > > now this two forms are also valid > > > if A { > BodyA > } > else if B { > BodyB > } > else { > BodyElse > } > > # or > > if A > { > BodyA > } > else if B > { > BodyB > } > else > { > BodyElse > } > > * now commas are required in lists, tuples and function arguments > * the try block is now try/catch/after to make it equal to erlang > * the receive block is now receive/else receive/after > * operator precedence made the same as erlang > * tests included to check that efene generates the same ast as > erlang, tests cover almost all the language > * now records can be declared in efene > * bit syntax made the same as erlang > > for a complete reference see the Language Reference > Participate > > a mailing list is available at librelist just send a mail to > efene@REDACTED to subscribe. > > as first mail you may send a hello world program in efene and present > yourself by saying your name, where you are, how did you heard about > efene and anything else you would like to say. > Resources > > * Website: http://marianoguerra.com.ar/efene > * Blog: http://efene.tumblr.com > * Central repo: http://github.com/marianoguerra/efene > * Tracker: http://github.com/marianoguerra/efene/issues > * Rosetta code page: http://rosettacode.org/wiki/Efene > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From mazen.harake@REDACTED Sun May 2 15:26:04 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Sun, 02 May 2010 15:26:04 +0200 Subject: [erlang-questions] terminal codes In-Reply-To: <4BDC9879.20807@eonblast.com> References: <4BDC2CF4.8030501@eonblast.com> <4BDC65B8.7000707@eonblast.com> <4BDC9879.20807@eonblast.com> Message-ID: <4BDD7D6C.4050808@erlang-solutions.com> I'm not sure this is 100% relevant to the discussion but if you like you can take a look at cecho which is an ncurses library for Erlang. You can set terminal modes (like cbreak mode and noecho) etc very easily, it also supports colors. Check it out here if interested: http://github.com/mazenharake/cecho /M On 01/05/2010 23:09, Henning Diedrich wrote: > -noshell does the trick! > > erl -s testtest run -s init stop -noshell > > Kind of logical in hindsight ... > > Beautiful, thanks! Can I see the snake game? > > Henning > > Stefan Marr wrote: >> Hi Henning: >> >> I have implemented a small snake game in Erlang using ANSI terminal >> control sequences, and it is started as follows: >> >> #!/bin/sh >> stty -echo -echok -icanon min 1 time 0 >> erl -pa ./ -run main start -run init stop -noshell >> stty echo echok icanon >> >> >> If I remember correctly, the stty is not relevant for you, it should >> set the shell to send all characters to Erlang, without awaiting a >> new-line. >> >> Hope that helps. >> >> Best regards >> Stefan >> >> On 01 May 2010, at 19:32, Henning Diedrich wrote: >> >>> This way of involing a program seems to prevent it as well. >>> >>> erl -s test run -s init stop >>> >>> What alternate way would allow control codes to get propagated >>> unchanched? >>> >>> Thanks! >>> Henning >>> >>> >>> ??? (Yau-Hsien Huang) wrote: >>>> I made a Erlang script and got output that seems correct. >>>> >>>> #!/usr/bin/env escript >>>> -compile(export_all). >>>> main([]) -> >>>> io:format("\e[1;36mabc\e[0;37m"). >>>> >>>> I checked it by pasting what printed into a telnet BBS and viewing it >>>> effect. It works. >>>> So the problem may shift to how to make Erlang shell support >>>> terminal color >>>> codes. >>>> >>>> On Sat, May 1, 2010 at 9:30 PM, Henning Diedrich >>>> wrote: >>>> >>>> >>>>> Can I print terminal color codes out of Erlang? >>>>> >>>>> E.g. \e[30m? >>>>> >>>>> The io: functions seem to force control codes to be printable. >>>>> >>>>> Thanks! >>>>> Henning >>>>> >>>>> >>>> >> >> >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From hd2010@REDACTED Sun May 2 15:33:41 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 15:33:41 +0200 Subject: Unix Epoch Message-ID: <4BDD7F35.1050002@eonblast.com> Hi, is there a better way to turn datetime into unix epoch seconds? unix_time({Date,Time}) -> UnixEpoch = {{1970,1,1},{0,0,0}}, calendar:datetime_to_gregorian_seconds({Date,Time}) - calendar:datetime_to_gregorian_seconds(UnixEpoch). Gregorian seconds down to year 0 might be a costly calculation I though. Thanks! Henning From roberto@REDACTED Sun May 2 15:38:05 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 2 May 2010 15:38:05 +0200 Subject: [erlang-questions] Unix Epoch In-Reply-To: <4BDD7F35.1050002@eonblast.com> References: <4BDD7F35.1050002@eonblast.com> Message-ID: i would suggest: unix_time() -> {M, S, _} = now(), M*1000000 + S. r. 2010/5/2 Henning Diedrich : > Hi, > > is there a better way to turn datetime into unix epoch seconds? > > unix_time({Date,Time}) -> > > ? UnixEpoch = {{1970,1,1},{0,0,0}}, > ? calendar:datetime_to_gregorian_seconds({Date,Time}) - > calendar:datetime_to_gregorian_seconds(UnixEpoch). > > Gregorian seconds down to year 0 might be a costly calculation I though. > > Thanks! > Henning > -- ------------------------------------------------------------------- Roberto Ostinelli CTO, WideTag Inc. - Realtime, Social, Green widetag.com skype: rostinelli twitter: ostinelli mobile: +39 335 6 100 22 6 From roberto@REDACTED Sun May 2 15:41:26 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 2 May 2010 15:41:26 +0200 Subject: [erlang-questions] Unix Epoch In-Reply-To: References: <4BDD7F35.1050002@eonblast.com> Message-ID: sorry, my bad. too fast in responding, must be my 38? fever. ignore my previous message. r. 2010/5/2 Roberto Ostinelli : > i would suggest: > > unix_time() -> > ? ? ? ?{M, S, _} = now(), > ? ? ? ?M*1000000 + S. > > r. From rapsey@REDACTED Sun May 2 15:45:53 2010 From: rapsey@REDACTED (Rapsey) Date: Sun, 2 May 2010 15:45:53 +0200 Subject: [erlang-questions] Unix Epoch In-Reply-To: <4BDD7F35.1050002@eonblast.com> References: <4BDD7F35.1050002@eonblast.com> Message-ID: Epoch seconds from 0 is a constant. So no need to calculate it every time. Sergej On Sun, May 2, 2010 at 3:33 PM, Henning Diedrich wrote: > Hi, > > is there a better way to turn datetime into unix epoch seconds? > > unix_time({Date,Time}) -> > > UnixEpoch = {{1970,1,1},{0,0,0}}, > calendar:datetime_to_gregorian_seconds({Date,Time}) - > calendar:datetime_to_gregorian_seconds(UnixEpoch). > > Gregorian seconds down to year 0 might be a costly calculation I though. > > Thanks! > Henning > From boris.okner@REDACTED Sun May 2 16:13:37 2010 From: boris.okner@REDACTED (Boris Okner) Date: Sun, 2 May 2010 10:13:37 -0400 Subject: [erlang-questions] Function to syntax tree In-Reply-To: References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> <4BDD4448.30106@gmail.com> Message-ID: Thanks Richard, So the sequence you're suggesting is: Function -> BEAM code -> Erlang source code -> AST? Still, assuming second step done by Decompiler, 3rd step by erl_scan and erl_parse, how do I get a BEAM code for anonymous function? On Sun, May 2, 2010 at 5:22 AM, Richard Carlsson wrote: > bokner wrote: > >> erl_scan and erl_parse can turn function represented by string into >> AST. For example, >> >> 1> {ok, S, _} = erl_scan:string("fun(X) -> X+1 end."), >> 1> {ok, P} = erl_parse:parse_exprs(S), >> 1> [F] = P, erl_syntax:type(F). >> fun_expr >> >> My question is: how do I do the same, but with the function itself >> instead string representation? I.e., >> >> F = fun(X) -> X +1 end, >> >> AST = , >> > > The "" would be something called > a Decompiler (see wikipedia for details), turning BEAM code > back into Erlang source code. See also: > http://www.erlang.org/pipermail/erlang-questions/2006-January/018810.html > > As far as I know, nobody has written a BEAM decompiler. > > /Richard > > From mononcqc@REDACTED Sun May 2 16:22:35 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 2 May 2010 10:22:35 -0400 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: <4BDD2F3B.2020605@eonblast.com> References: <4BDD2F3B.2020605@eonblast.com> Message-ID: Untested: milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro), Mega >= 0, Sec >= 0, Micro >= 0 -> Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000); milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro) -> erlang:error(negative_input). Notice that I used erlang:error/1 rather than throw/1. The reason is that throw/1 is meant to be used mainly for control flow or very common exceptions you know how to fix, and should never be seen by the programmer -- they ought to be restricted to the module where they happen. Errors represent a piece of code you want to fail but you can't expect to fix without changing the code. On Sun, May 2, 2010 at 3:52 AM, Henning Diedrich wrote: > Hi folks, > > is there a way to add detail to a function_clause error messages? Be it > guards or patterns that don't match? > > On a crash, I would like to clarify what guard failed, or even warn the > programmer that this guard exists when it may not be intuitive. > > 1 > > This is written, I believe, as it should: > > milli_epoch({Mega, Sec, Micro}) when > is_integer(Mega), is_integer(Sec), is_integer(Micro), > Mega >= 0, Sec >= 0, Micro >= 0 -> > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). > > 2 > > This fails the way I would like it to, giving "negative_input" as reason. > > But it is written wrong for two (2) counts I believe (I am NOT trying to > sneak in a point about ifs): > > milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), > is_integer(Micro) -> > > if (Mega >= 0), (Sec >= 0), (Micro >= 0) -> ok; > true -> throw(negative_input) end, > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). > > Thank you very much for taking a look, > > Henning > > > From boris.okner@REDACTED Sun May 2 17:11:33 2010 From: boris.okner@REDACTED (Boris Okner) Date: Sun, 2 May 2010 11:11:33 -0400 Subject: [erlang-questions] Function to syntax tree In-Reply-To: <4BDD935A.3050106@gmail.com> References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> <4BDD4448.30106@gmail.com> <4BDD935A.3050106@gmail.com> Message-ID: Yes, I read beam_lib documentation. I still don't see how can I use it to get BEAM code for the anonymous function dynamically created during run-time. Boris On Sun, May 2, 2010 at 10:59 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > Boris Okner wrote: > >> Thanks Richard, >> >> So the sequence you're suggesting is: >> >> Function -> BEAM code -> Erlang source code -> AST? >> >> Still, assuming second step done by Decompiler, 3rd step by erl_scan and >> erl_parse, how do I get a BEAM code for anonymous function? >> > > Look at the documentation for the beam_lib module in stdlib. Also, > if the code has been compiled with debug information, it is almost > trivial to reconstruct the source code (see beam_lib for details). > > /Richard > From hd2010@REDACTED Sun May 2 17:22:55 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 17:22:55 +0200 Subject: [erlang-questions] Unix Epoch In-Reply-To: References: <4BDD7F35.1050002@eonblast.com> Message-ID: <4BDD98CF.6000207@eonblast.com> That's what I would hope for, this leaves datetime_to_gregorian_seconds({Date,Time}) which is also, and needlessly, calculated back to year 0. Rapsey wrote: > Epoch seconds from 0 is a constant. So no need to calculate it every time. > > > Sergej > > On Sun, May 2, 2010 at 3:33 PM, Henning Diedrich wrote: > > >> Hi, >> >> is there a better way to turn datetime into unix epoch seconds? >> >> unix_time({Date,Time}) -> >> >> UnixEpoch = {{1970,1,1},{0,0,0}}, >> calendar:datetime_to_gregorian_seconds({Date,Time}) - >> calendar:datetime_to_gregorian_seconds(UnixEpoch). >> >> Gregorian seconds down to year 0 might be a costly calculation I though. >> >> Thanks! >> Henning >> >> > > From hd2010@REDACTED Sun May 2 17:26:25 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 17:26:25 +0200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: References: <4BDD2F3B.2020605@eonblast.com> Message-ID: <4BDD99A1.6090906@eonblast.com> Thanks a lot, also for clarifying the use of throw and error on the occasion! Henning Fred Hebert wrote: > Untested: > > milli_epoch({Mega, Sec, Micro}) when > is_integer(Mega), is_integer(Sec), > is_integer(Micro), > Mega >= 0, Sec >= 0, Micro >= 0 -> > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000); > milli_epoch({Mega, Sec, Micro}) when > is_integer(Mega), is_integer(Sec), is_integer(Micro) -> > erlang:error(negative_input). > > Notice that I used erlang:error/1 rather than throw/1. The reason is > that throw/1 is meant to be used mainly for control flow or very > common exceptions you know how to fix, and should never be seen by the > programmer -- they ought to be restricted to the module where they > happen. Errors represent a piece of code you want to fail but you > can't expect to fix without changing the code. > > On Sun, May 2, 2010 at 3:52 AM, Henning Diedrich > wrote: > > Hi folks, > > is there a way to add detail to a function_clause error messages? > Be it guards or patterns that don't match? > > On a crash, I would like to clarify what guard failed, or even > warn the programmer that this guard exists when it may not be > intuitive. > > 1 > > This is written, I believe, as it should: > > milli_epoch({Mega, Sec, Micro}) when > is_integer(Mega), is_integer(Sec), is_integer(Micro), > Mega >= 0, Sec >= 0, Micro >= 0 -> > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). > > 2 > > This fails the way I would like it to, giving "negative_input" as > reason. > > But it is written wrong for two (2) counts I believe (I am NOT > trying to sneak in a point about ifs): > > milli_epoch({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro) -> > > if (Mega >= 0), (Sec >= 0), (Micro >= 0) -> ok; > true -> throw(negative_input) end, > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). > > Thank you very much for taking a look, > > Henning > > > From tony.arcieri@REDACTED Sun May 2 19:55:33 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Sun, 2 May 2010 11:55:33 -0600 Subject: [erlang-questions] Function to syntax tree In-Reply-To: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> Message-ID: On Sat, May 1, 2010 at 9:03 PM, bokner wrote: > My question is: how do I do the same, but with the function itself > instead string representation? I.e., > > F = fun(X) -> X +1 end, > > AST = , > fun_expr = erl_syntax:type(AST). > This maybe? Fun = fun() -> 2 + 2 end, {env, Env} = erlang:fun_info(Fun, env), [Abs|_] = lists:reverse(Env) Note that will get you to the Erlang abstract format representation... you'll need to convert that to AST: AST = erl_syntax:abstract(Abs) -- Tony Arcieri Medioh! A Kudelski Brand From boris.okner@REDACTED Sun May 2 20:31:18 2010 From: boris.okner@REDACTED (Boris Okner) Date: Sun, 2 May 2010 14:31:18 -0400 Subject: [erlang-questions] Function to syntax tree In-Reply-To: References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> Message-ID: Thanks a lot Tony, Really got me a lot closer. The resulting AST is of type list though, but I can now see how to transform it to the function AST, or I may even be able to use it as is. Boris On Sun, May 2, 2010 at 1:55 PM, Tony Arcieri wrote: > On Sat, May 1, 2010 at 9:03 PM, bokner wrote: > >> My question is: how do I do the same, but with the function itself >> instead string representation? I.e., >> >> F = fun(X) -> X +1 end, >> >> AST = , >> fun_expr = erl_syntax:type(AST). >> > > This maybe? > > Fun = fun() -> 2 + 2 end, > {env, Env} = erlang:fun_info(Fun, env), > [Abs|_] = lists:reverse(Env) > > Note that will get you to the Erlang abstract format representation... > you'll need to convert that to AST: > > AST = erl_syntax:abstract(Abs) > > -- > Tony Arcieri > Medioh! A Kudelski Brand > From luismarianoguerra@REDACTED Sun May 2 21:36:37 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sun, 2 May 2010 16:36:37 -0300 Subject: [erlang-questions] [ANN] efene 0.5 - two languages for the erlang VM - released In-Reply-To: <4BDD3E64.9030907@eonblast.com> References: <4BDD3E64.9030907@eonblast.com> Message-ID: On Sun, May 2, 2010 at 5:57 AM, Henning Diedrich wrote: > Did I overlook that? What syntax are you using for pattern matching, e.g. > > erl_integer(<>=X), E >= ?LONG_MIN, E =< ?LONG_MAX -> > ? ? E. the second part is a guard? there is a , after the ) If I understand correctly that would be erl_integer(<[E:64/signed]>=X) when E >= ?LONG_MIN or E =< ?LONG_MAX E the bit syntax is almost the same, the << and >> are changed to <[ and ]> because in efene bsl and bsr are << and >> like in C et al. note: efene doesn't support -define yet, so that constants should be defined in another way. From luismarianoguerra@REDACTED Sun May 2 21:39:01 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sun, 2 May 2010 16:39:01 -0300 Subject: [erlang-questions] [ANN] efene 0.5 - two languages for the erlang VM - released In-Reply-To: <4BDD3484.6020603@eonblast.com> References: <4BDD3484.6020603@eonblast.com> Message-ID: On Sun, May 2, 2010 at 5:15 AM, Henning Diedrich wrote: > Very exciting --- can I ask the others what they think is necessary > regarding tests to find efene reliable? right now I'm testing that a given efene expression generates the same abstract form as erlang, you can look the tests here: http://github.com/marianoguerra/efene/tree/master/test/ for example for expressions: http://github.com/marianoguerra/efene/blob/master/test/exprs.erl if anyone has another way to test a language for reliability that would be nice to know. Mariano, From hd2010@REDACTED Sun May 2 23:20:46 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 02 May 2010 23:20:46 +0200 Subject: [erlang-questions] Unix Epoch In-Reply-To: References: <4BDD7F35.1050002@eonblast.com> Message-ID: <4BDDECAE.5080504@eonblast.com> Wow, thanks, is this Erlang compiler/VM source? Tony Finch wrote: > On Sun, 2 May 2010, Henning Diedrich wrote: > >> Gregorian seconds down to year 0 might be a costly calculation I though. >> > > No more expensive than calculating from 1970. The only difference is the > offset constant. The following code follows the conventions in the book > "Calendrical Calculations" though it is somewhat optimised. > > int gregorian_date_to_day_number(int y, int m, int d) { > // Move Jan and Feb to previous year so leap days fall at the end > // and number months Mar=4 - Feb=15 so the 5 month cycle fits. > if (m > 2) m += 1; > else m += 13, y -= 1; > return > // Days in this month. > + d > // Days in this year before the current month, using repeating > // cycle of 5 months Mar - Jul, Aug - Dec, Jan - (truncated). > + m*153/5 > // Days in previous years according to 4-year Julian cycle. > + y*1461/4 > // Gregorian correction. > - y/100 + y/400 > // Epoch offset so that Mon 0001-01-01 is R.D. 1. > - 428; > // ISO 8601 numbers days of the week Mon=1 - Sun=7 > // Wed 1858-11-17 is R.D. 678576 and MJD 0 > // Thu 1970-01-01 is R.D. 719163 > } > > int gregorian_time_to_unix_time(int y, int m, int d, int H, int M, int S) { > return 86400 * (gregorian_date_to_day_number(y, m, d) - 719163) > + H * 1440 + M * 60 + S; > } > > Tony. > From ok@REDACTED Mon May 3 01:21:26 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 3 May 2010 11:21:26 +1200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: <4BDD2F3B.2020605@eonblast.com> References: <4BDD2F3B.2020605@eonblast.com> Message-ID: On May 2, 2010, at 7:52 PM, Henning Diedrich wrote: > Hi folks, > > is there a way to add detail to a function_clause error messages? Be > it guards or patterns that don't match? > > On a crash, I would like to clarify what guard failed, or even warn > the programmer that this guard exists when it may not be intuitive. Why? What is the programmer supposed to do about it? More to the point, what is the *program* supposed to do about it? > > 1 > > This is written, I believe, as it should: > > milli_epoch({Mega, Sec, Micro}) when > is_integer(Mega), is_integer(Sec), is_integer(Micro), > Mega >= 0, Sec >= 0, Micro >= 0 -> > Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). In a strictly typed language like Ada, one would arrange for it to be _impossible_ to construct a record that didn't pass the test. Is this something the Dialyzer can help with? Why not separate the task into two issues: (A) do the main job (B) diagnose the problem As a general rule, doing this means that someone can call the (B) function for some value long before calling the (A) function. The (A) function can then be streamlined. For example, the Mega >= 0, Sec >= 0, Micro >= 0 tests can be removed. Not only is the function perfectly safe without them, but one expects to be able to do arithmetic on timestamps, and that means that if {M,S,U} is valid, one also *expects* that {-M,-S,-U} should be valid. (M,S,U should all be non-negative or all be non-positive, but that's not what the code above tests.) From ok@REDACTED Mon May 3 01:33:49 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 3 May 2010 11:33:49 +1200 Subject: [erlang-questions] closure on function argument In-Reply-To: References: <4BDC7512.60604@tmit.bme.hu> Message-ID: On May 2, 2010, at 8:56 AM, Robert Virding wrote: > The variables in the arguments of a fun, Y in your example, are always > "fresh" variables and never imported from the funs environment. For me personally, this is perhaps THE most confusing thing about Erlang. I have *no* problem with the same rule in ML or Haskell because they apply it *consistently*: every pattern everywhere introduces fresh variables, it is never ever possible in those languages for a variable occurrence in a pattern to be an applied occurrence; all pattern variables are binding occurrences. This applies to OCaml and Clean as well. But Erlang does NOT apply this rule consistently. Variables in a pattern of a 'case' are *not* fresh, they *may* be applied occurrences. And even in a fun, *some* variable occurrences may be applied ones: fun (Y,Y) -> ... end. It's especially confusing because variables in fun bodies and guards are imported as usual, so they are clearly in scope. It would be *so* nice if ALL variables in Erlang followed exactly the same rules. Here's a little example. Suppose someone has case E of P -> B end but doesn't want any variable bindings in B to be exported. The obvious thing to do is to rewrite it as (fun (P) -> B end)(E) but because Erlang is inconsistent here, that does not work. (You can patch around it with fun, case, and a new variable, but that has problems of its own.) I have never been able to see any merit in this rule; its effect on me is to make it harder to write funs because I always have to watch out for cases where the expected thing won't happen and it has to be patched around. From ulf.wiger@REDACTED Mon May 3 08:53:02 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 03 May 2010 08:53:02 +0200 Subject: [erlang-questions] Function to syntax tree In-Reply-To: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> Message-ID: <4BDE72CE.8010904@erlang-solutions.com> Perhaps this code snippet might be of some help? http://www.erlang.org/cgi-bin/ezmlm-cgi/4/31984 I have not tried to see whether it still works, though. BR, Ulf W bokner wrote: > Hi, > > erl_scan and erl_parse can turn function represented by string into > AST. For example, > > 1> {ok, S, _} = erl_scan:string("fun(X) -> X+1 end."), > 1> {ok, P} = erl_parse:parse_exprs(S), > 1> [F] = P, erl_syntax:type(F). > fun_expr > > My question is: how do I do the same, but with the function itself > instead string representation? I.e., > > F = fun(X) -> X +1 end, > > AST = , > fun_expr = erl_syntax:type(AST). > > Thanks, > Boris > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From hd2010@REDACTED Mon May 3 13:53:33 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 03 May 2010 13:53:33 +0200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: References: <4BDD2F3B.2020605@eonblast.com> Message-ID: <4BDEB93D.8070008@eonblast.com> Thank you for the advice Richard! The function is to be called often and directly on non-tested data coming in over the network. So at first sight, splitting up seems to bloat things instead of making them clearer in this case. There is no moment in time, long up front, when the diagnosis should be done. I'll give splitting up a try to learn how that looks. > Why? What is the programmer supposed to do about it? > More to the point, what is the *program* supposed to do about it? 1) Discard the data he/it is trying to parse and tell the data source that it was no good, and why. 2) If the data was self-generated, e.g. for testing, the programmer to use this function may make assumptions that are wrong and the programmer to program this function (me) wants a way to inform him in case he runs into trouble. By providing more information than function_clause. Especially if I can imagine what the wrong assumption may be. As you are pointing out, why wouldn't I be allowed to feed negative values in. [...] > Why not separate the task into two issues: > (A) do the main job > (B) diagnose the problem > As a general rule, doing this means that someone can call the (B) > function for some value long before calling the (A) function. > > The (A) function can then be streamlined. Thanks, Henning From seancribbs@REDACTED Mon May 3 14:59:01 2010 From: seancribbs@REDACTED (Sean Cribbs) Date: Mon, 3 May 2010 08:59:01 -0400 Subject: [erlang-questions] Function to syntax tree In-Reply-To: <4BDE72CE.8010904@erlang-solutions.com> References: <10593c09-d391-4dd0-9e11-90aeb5fcda2a@s29g2000yqd.googlegroups.com> <4BDE72CE.8010904@erlang-solutions.com> Message-ID: I did something similar a couple months ago, but taking a fun() and turning it into a module function with smerl: http://seancribbs.com/tech/2010/01/12/smerl-for-awesome/ On Mon, May 3, 2010 at 2:53 AM, Ulf Wiger wrote: > > Perhaps this code snippet might be of some help? > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/31984 > > I have not tried to see whether it still works, though. > > BR, > Ulf W > > bokner wrote: >> >> Hi, >> >> erl_scan and erl_parse can turn function represented by string into >> AST. For example, >> >> 1> {ok, S, _} = erl_scan:string("fun(X) -> X+1 end."), >> 1> {ok, P} = erl_parse:parse_exprs(S), >> 1> [F] = P, erl_syntax:type(F). >> fun_expr >> >> My question is: how do I do the same, but ?with the function itself >> instead string representation? I.e., >> >> F = fun(X) -> X +1 end, >> >> AST = , >> fun_expr = erl_syntax:type(AST). >> >> Thanks, >> Boris >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Ulf Wiger > CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd > http://www.erlang-solutions.com > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From attila.r.nohl@REDACTED Mon May 3 17:14:48 2010 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 3 May 2010 17:14:48 +0200 Subject: Behaviour style question Message-ID: Hello! I have a behaviour implemented by 6 modules. The functions of these modules are called by an other module (let's call it manager module). Then it turned out that one of the 6 modules needs a slightly different interface than the other five, it needs an extra function. Of course, the manager module has to be modified, there's a case statement in the code. My question is what to do with the behaviour? If I add the extra function to the behaviour, I'll get compilation warnings in 5 modules and that's generally a bad idea, because due to the these warnings I might not notice more important warnings. I can implement the extra function in all 5 modules like get_alarm_type_data(_) -> erlang:error(this_should_not_be_called); but I don't like this unnecessary code (I'd get a function_clause without this and that's basically the same result) and I especially don't like it in 5 files. My gut feeling is that the 6th module with the extra function has a different behaviour, so there should be a new behaviour for this module (even if it's only implemented by one module). In this case I should add to the documentation of the manager module that the Module parameter to its functions must implement either the original or the "extended" behaviour. Any other ideas? From alessandro.sivieri@REDACTED Mon May 3 17:26:33 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Mon, 3 May 2010 17:26:33 +0200 Subject: [erlang-questions] Behaviour style question In-Reply-To: References: Message-ID: Given the fact that I have no idea in how to write a behaviour (I have only implemented the existing ones), so my opinion may be completely wrong: you may have to do like the gen_server one, in which the format_status callback function is optional; I think this should be the best way, even if I don't know how to realize it... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From comptekki@REDACTED Mon May 3 18:05:01 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 3 May 2010 10:05:01 -0600 Subject: why this error? Message-ID: {ok,{Status,Header,Body}}=http:request(URL). ** exception error: no match of right hand side value {ok,{{"HTTP/1.1",200,"OK"}, [{"connection","Keep-Alive"}, {"date","Mon, 03 May 2010 15:57:33 GMT"}, {"server", "...."}, {"content-length","202"}, {"content-type","text/html; charset=UTF-8"}, {"set-cookie","SESSID=00000011111"}, {"keep-alive","timeout=15, max=100"}], "\n...."}} It looks right to me but it is throwing a right hand not match error. Why? There is an "ok" which matches ok on the left. There is a next section of { {}, [], "" } which seems to match {Status,Header,Body} of three items. What am I missing? -wes From erlangy@REDACTED Mon May 3 18:13:57 2010 From: erlangy@REDACTED (Michael McDaniel) Date: Mon, 3 May 2010 09:13:57 -0700 Subject: [erlang-questions] why this error? In-Reply-To: References: Message-ID: <20100503161356.GW5061@delora.autosys.us> On Mon, May 03, 2010 at 10:05:01AM -0600, Wes James wrote: > {ok,{Status,Header,Body}}=http:request(URL). > ** exception error: no match of right hand side value > {ok,{{"HTTP/1.1",200,"OK"}, > > > What am I missing? > > -wes > ___________________________________________________________________________ presumably you are doing this from the shell, see how this works ... f(Status), f(Header), f(Body), {ok,{Status,Header,Body}}=http:request(URL). ~Michael From erlangy@REDACTED Mon May 3 18:15:50 2010 From: erlangy@REDACTED (Michael McDaniel) Date: Mon, 3 May 2010 09:15:50 -0700 Subject: [erlang-questions] why this error? In-Reply-To: <20100503161356.GW5061@delora.autosys.us> References: <20100503161356.GW5061@delora.autosys.us> Message-ID: <20100503161550.GX5061@delora.autosys.us> On Mon, May 03, 2010 at 09:13:56AM -0700, Michael McDaniel wrote: > On Mon, May 03, 2010 at 10:05:01AM -0600, Wes James wrote: > > {ok,{Status,Header,Body}}=http:request(URL). > > ** exception error: no match of right hand side value > > {ok,{{"HTTP/1.1",200,"OK"}, > > > > > > > What am I missing? > > > > -wes > > > ___________________________________________________________________________ > > > presumably you are doing this from the shell, see how this works ... > > > f(Status), f(Header), f(Body), {ok,{Status,Header,Body}}=http:request(URL). > > > ~Michael ___________________________________________________________________________ oh, and help(). in the shell to see what it is ~M From mabrek@REDACTED Mon May 3 17:23:48 2010 From: mabrek@REDACTED (mabrek) Date: Mon, 03 May 2010 19:23:48 +0400 Subject: low overhead profiler for erlang? Message-ID: <4BDEEA84.2030101@gmail.com> Hello. Is there a way to do profiling with low overhead? I tried eprof and fprof, in both cases profiling slowed down my application 10 times. It was able to serve 40 http requests per second without profiling but when I enabled profiling speed dropped down to 4 requests per second. It seems that such a big overhead skews results of profiling adding time to frequently called functions. I knew that the most time consuming part of my application was database queries (I proved that later by replacing external SQL database with mnesia RAM table), but profiler failed to show it. Frequently called xmerl xml escaping functions were above all in a sorted by accumulated time results. Regards, Anton Lebedevich. From comptekki@REDACTED Mon May 3 19:16:14 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 3 May 2010 11:16:14 -0600 Subject: & is not getting replaced in re:replace Message-ID: Is there something special with "&" in erlang. re:replace(T, "<", "&", [{return, list}, global]). "<" I noticed that a string I had was not getting changed and it came down to the above re:replace of "<" with "&" return "<". Why? thx, -wes From comptekki@REDACTED Mon May 3 19:26:46 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 3 May 2010 11:26:46 -0600 Subject: & is not getting replaced in re:replace In-Reply-To: References: Message-ID: On Mon, May 3, 2010 at 11:16 AM, Wes James wrote: > Is there something special with "&" in erlang. > > > re:replace(T, "<", "&", [{return, list}, global]). > "<" > > I noticed that a string I had was not getting changed and it came down > to the above re:replace of "<" with "&" return "<". ?Why? OK - I found it here: http://www.erlang.org/doc/man/re.html From mononcqc@REDACTED Mon May 3 19:26:52 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 3 May 2010 13:26:52 -0400 Subject: [erlang-questions] & is not getting replaced in re:replace In-Reply-To: References: Message-ID: That's a standard regex character. The & usually hints at something like a reference to a capture in the regular expression. You need to escape it: 1> re:replace("hell wrote: > Is there something special with "&" in erlang. > > > re:replace(T, "<", "&", [{return, list}, global]). > "<" > > I noticed that a string I had was not getting changed and it came down > to the above re:replace of "<" with "&" return "<". Why? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dale@REDACTED Mon May 3 19:28:23 2010 From: dale@REDACTED (Dale Harvey) Date: Mon, 3 May 2010 18:28:23 +0100 Subject: [erlang-questions] & is not getting replaced in re:replace In-Reply-To: References: Message-ID: from http://erldocs.com/R13B04/stdlib/re.html?search=re%20replace&i=3#replace/3 To insert an & or \ in the result, precede it with a \. On 3 May 2010 18:16, Wes James wrote: > Is there something special with "&" in erlang. > > > re:replace(T, "<", "&", [{return, list}, global]). > "<" > > I noticed that a string I had was not getting changed and it came down > to the above re:replace of "<" with "&" return "<". Why? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Mon May 3 20:31:48 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 3 May 2010 12:31:48 -0600 Subject: httpc:store_cookies stopped working Message-ID: All morning I've been doing: 14> httpc:reset_cookies(). ok 15> httpc:store_cookies([{"set-cookie","TESTID=set"}], URL). ok 16> httpc:which_cookies(). [{session_cookies,[]}] But now notice above that the cookies aren't storing anymore. Any idea why? Before I had items in the [] to the right of the session_cookies. I have run inets:start() and ssl:start() before I started doing these. thx, -wes From comptekki@REDACTED Mon May 3 20:45:27 2010 From: comptekki@REDACTED (Wes James) Date: Mon, 3 May 2010 12:45:27 -0600 Subject: httpc:store_cookies stopped working In-Reply-To: References: Message-ID: On Mon, May 3, 2010 at 12:31 PM, Wes James wrote: > All morning I've been doing: > > 14> httpc:reset_cookies(). > ok > 15> httpc:store_cookies([{"set-cookie","TESTID=set"}], URL). > ok > 16> httpc:which_cookies(). > [{session_cookies,[]}] > > But now notice above that the cookies aren't storing anymore. ?Any > idea why? ?Before I had items in the [] to the right of the > session_cookies. > > I have run inets:start() and ssl:start() before I started doing these. Hmm - I had to do this again: http:set_options([{cookies, enabled}]). and now it works. -wes From org.erlang@REDACTED Mon May 3 20:53:12 2010 From: org.erlang@REDACTED (org.erlang@REDACTED) Date: Mon, 3 May 2010 18:53:12 +0000 Subject: 'new' SSL API crash Message-ID: <20100503185312.GB75944@logik.internal.network> Hello. Is the 'ssl_new' module known to be broken, currently? I've tried using the 'old' ssl module but it seems to have no way to actually fail SSL handshakes for non-verified connections and the like, so I'm forced to use the new implementation. However, I can't get it to do anything without crashing. I know the certificates are valid as I've tested them using the openssl s_client and s_server command line utilities. Test program: -module (sslserv_new). -export ([start/0]). server_ssl_settings () -> [ %% Socket options. {active, false}, {reuseaddr, true}, %% SSL options. {cacertfile, "TEST_CA/ca-cert.pem"}, {certfile, "TEST_CA/hosts/pacifico_server/cert.pem"}, {keyfile, "TEST_CA/hosts/pacifico_server/key.pem"}, {ssl_imp, new}, {verify, verify_peer}, {depth, 1}, {ciphers, ssl:cipher_suites()}, {reuse_sessions, false} ]. start() -> io:format ("ssl:listen\n"), {ok, Socket} = ssl:listen (10000, server_ssl_settings ()), io:format ("ssl:transport_accept\n"), {ok, Client_Socket} = ssl:transport_accept (Socket), io:format ("ssl:ssl_accept\n"), case ssl:ssl_accept (Client_Socket) of ok -> io:format ("ssl:ssl_accept: accepted\n"), io:format ("ssl:close\n"), ok = ssl:close (Client_Socket); {error, Reason} -> io:format ("ssl:ssl_accept: error ~w\n", [Reason]) end, io:format ("ssl:close\n"), ok = ssl:close (Socket). -- Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) application:start (sasl), application:start (crypto), application:start (ssl), -- 1> ssl:versions(). [{ssl_app,"3.10.7"}, {supported,[tlsv1,sslv3]}, {available,[tlsv1,sslv3]}] 2> sslserv_new:start(). ssl:listen ssl:transport_accept ssl:ssl_accept Then, connecting using 'openssl s_client' using a known, verifiable test certificate: ** exception exit: {{function_clause,[{pubkey_cert,select_extension, [{2,5,29,35},asn1_NOVALUE]}, {pubkey_cert,issuer_id,2}, {ssl_certificate,certificate_chain,4}, {ssl_handshake,certificate,3}, {ssl_connection,certify_server,1}, {ssl_connection,server_certify_and_key_exchange,1}, {ssl_connection,do_server_hello,2}, {lists,foldl,3}]}, {gen_fsm,sync_send_all_state_event, [<0.60.0>,started,infinity]}} in function gen_fsm:sync_send_all_state_event/3 in call from ssl:ssl_accept/2 in call from sslserv_new:start/0 3> =ERROR REPORT==== 3-May-2010::15:57:29 === ** State machine <0.60.0> terminating ** Last event in was {ssl_tls,undefined,22, {3,1}, <<1,0,0,133,3,1,0,93,0,0,0,32,0,0,57,0,0,56,0,0, 53,0,0,136,0,0,135,0,0,132,0,0,22,0,0,19,0,0, 10,7,0,192,0,0,51,0,0,50,0,0,47,0,0,69,0,0,68, 0,0,65,3,0,128,0,0,5,0,0,4,1,0,128,0,0,21,0,0, 18,0,0,9,6,0,64,0,0,20,0,0,17,0,0,8,0,0,6,4,0, 128,0,0,3,2,0,128,198,211,40,132,76,120,105, 20,171,188,10,216,42,133,0,122,173,212,152, 167,161,137,84,183,0,215,233,12,153,221,99,235>>} (for all states) ** When State == hello ** Data == {state,server, {#Ref<0.0.0.70>,<0.35.0>}, gen_tcp,tcp,tcp_closed,"localhost",10000,#Port<0.1154>, {ssl_options,[],verify_none,#Fun,false, false,1,"TEST_CA/hosts/pacifico_server/cert.pem", "TEST_CA/hosts/pacifico_server/key.pem",undefined, undefined,"TEST_CA/ca-cert.pem", [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], #Fun,false,[]}, {socket_options,list,0,0,0,false}, {connection_states, {connection_state, {security_parameters,undefined,0,0,0,undefined, undefined,undefined,undefined,0,undefined,0, undefined,undefined,undefined,undefined}, undefined,undefined,undefined,1}, {connection_state, {security_parameters,undefined,0,undefined, undefined,undefined,undefined,undefined, undefined,undefined,undefined,undefined, undefined,undefined, <<75,222,242,105,194,191,18,141,171,244,247, 203,234,237,111,11,152,119,181,103,91,155, 92,85,84,17,57,121,20,164,73,110>>, undefined}, undefined,undefined,undefined,undefined}, {connection_state, {security_parameters,undefined,0,0,0,undefined, undefined,undefined,undefined,0,undefined,0, undefined,undefined,undefined,undefined}, undefined,undefined,undefined,0}, {connection_state, {security_parameters,undefined,0,undefined, undefined,undefined,undefined,undefined, undefined,undefined,undefined,undefined, undefined,undefined, <<75,222,242,105,194,191,18,141,171,244,247, 203,234,237,111,11,152,119,181,103,91,155, 92,85,84,17,57,121,20,164,73,110>>, undefined}, undefined,undefined,undefined,undefined}}, <<>>,<<>>, {{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, 0,0,0,0,0>>, <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116,97, 120,95,116,111,111,108,115,45,49,46,54,46,52,47,101, 98,105,110,47,115,115,108,95,104,97,110,100,115,104, 97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0,0,0, 185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}, {<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, 0,0,0,0,0>>, <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116, 97,120,95,116,111,111,108,115,45,49,46,54,46,52,47, 101,98,105,110,47,115,115,108,95,104,97,110,100,115, 104,97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0, 0,0,185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}}, [], <<48,130,3,93,48,130,2,69,2,1,1,48,13,6,9,42,134,72,134, 247,13,1,1,5,5,0,48,129,136,49,17,48,15,6,3,85,4,10,19, 8,80,65,67,73,70,73,67,79,49,20,48,18,6,3,85,4,11,20,11, 112,97,99,105,102,105,99,111,95,99,97,49,36,48,34,6,9, 42,134,72,134,247,13,1,9,1,22,21,112,97,99,105,102,105, 99,111,95,99,97,64,108,111,99,97,108,104,111,115,116,49, 10,48,8,6,3,85,4,7,19,1,46,49,10,48,8,6,3,85,4,8,19,1, 46,49,11,48,9,6,3,85,4,6,19,2,90,90,49,18,48,16,6,3,85, 4,3,19,9,108,111,99,97,108,104,111,115,116,48,30,23,13, 49,48,48,53,48,50,49,51,52,57,52,51,90,23,13,50,48,48, 52,50,57,49,51,52,57,52,51,90,48,96,49,11,48,9,6,3,85,4, 6,19,2,90,90,49,10,48,8,6,3,85,4,8,19,1,46,49,17,48,15, 6,3,85,4,10,19,8,80,65,67,73,70,73,67,79,49,24,48,22,6, 3,85,4,11,20,15,112,97,99,105,102,105,99,111,95,115,101, 114,118,101,114,49,24,48,22,6,3,85,4,3,20,15,112,97,99, 105,102,105,99,111,95,115,101,114,118,101,114,48,130,1, 34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,130,1,15, 0,48,130,1,10,2,130,1,1,0,240,234,138,84,118,47,107,232, 150,182,200,67,1,35,15,48,208,88,231,11,213,21,249,110, 226,145,33,249,255,250,114,58,255,247,130,249,140,60, 108,201,2,98,26,254,16,213,173,238,140,201,39,75,7,97, 209,120,94,80,69,11,72,117,162,83,84,194,57,232,106,19, 2,196,52,255,102,220,178,30,82,85,10,96,118,20,104,238, 54,214,183,157,110,205,247,220,236,34,209,225,95,113, 244,195,193,56,177,196,204,248,203,210,172,236,124,75, 60,246,172,183,76,9,253,155,20,101,63,66,12,109,213,186, 167,95,2,197,100,120,22,94,247,229,2,22,95,54,216,42,78, 230,204,144,123,218,153,113,128,155,236,88,228,171,169, 59,165,2,122,196,149,208,179,249,5,86,38,39,217,79,193, 7,121,41,193,201,3,43,109,225,62,195,67,173,248,3,245, 81,210,197,193,236,3,91,150,57,142,97,34,18,138,104,212, 252,188,6,165,221,221,243,166,115,40,247,179,191,163, 127,190,211,153,161,229,113,220,178,216,48,240,116,133, 103,20,107,225,88,214,163,63,233,4,175,50,61,6,22,240, 78,107,145,2,3,1,0,1,48,13,6,9,42,134,72,134,247,13,1,1, 5,5,0,3,130,1,1,0,103,56,251,221,227,238,147,176,66,136, 67,183,114,184,232,52,77,105,35,127,218,87,140,246,244, 131,178,205,126,183,13,38,12,90,3,172,190,31,142,5,170, 202,43,229,222,118,97,167,201,150,182,54,41,67,173,234, 202,139,219,152,255,34,15,191,247,240,37,69,210,46,137, 148,86,105,182,112,77,238,106,8,115,135,239,117,148,12, 71,65,61,149,149,238,4,8,48,118,236,135,158,183,156,215, 132,122,46,139,35,81,172,60,217,218,157,198,183,10,142, 41,67,186,92,144,238,232,144,223,150,33,7,141,63,164, 230,173,145,48,36,152,147,32,83,156,37,23,191,250,58,51, 26,228,110,156,248,226,26,247,42,10,180,228,98,202,249, 31,223,122,26,83,94,47,134,135,202,150,140,201,178,178, 137,77,116,240,36,174,17,44,221,75,61,53,196,213,107,79, 230,160,158,202,223,113,180,19,11,255,19,247,161,211, 221,185,204,232,155,16,207,175,83,7,237,175,220,121,90, 170,96,198,200,43,146,250,75,86,97,65,60,250,211,255, 104,133,172,217,210,30,144,86,10,90,96,157,85,79,59,249, 206,138>>, {session,undefined,undefined,undefined,undefined, undefined,false,undefined}, 20499,ssl_session_cache,undefined,undefined,false, undefined,undefined, {'RSAPrivateKey','two-prime', 30412849349635586059878763176680338042806370928303169072580788733024438138591624114344661613699569307282187776490585714522517050565529617950655734681469324828571314711428867434319920530980711392758700590981596083356704738486321722811445451684075618202284684847604912244382636109829288466866694913244559971697849183994297490539908862341588923396172960090070282527346558060132865021448687169667310568206513992904712186265982889729037153800223846493957161193620543020688239671219777230989849538520312806141702778165525520364118176466217430310054602386812099554335308017341261109213665732076271625040081260558423830719377, 65537, 15199231804150546259658042484791966551963578848813856243560714914822915640833292103958537953118116384964424618816792863691594075974988046092098009948916868274565455708125634299346634071611627633962886925804052924579735878963676936497600941132620179791254257637877896314147191365262961154085247394502328039931767245176290407517156245147167225384726350170243685109653662879990481102526168621741076751484642682168826381020813101510533444174790316691177259394914972358200408962095948054841259799747816506752270496315220170738960924479086469481940145231690103702610522190880399648129761477972116477505211965989493531309937, 176191409302436409962613455746475864610360333729834662591534394409663736050830385559280211933901474704302550960882948381498437634097559237504384892632519298195621737913663090630551501287500290638791916696994910572811737440764017948768162498168387518624603640493626338734757174408297008712871342547046062968773, 172612555118571442807091000766318626155131399021717029980434951225715253087609998682422104752170804119306391847675331885937326037489735337859546704781797012286523769446860097366850215779124291431114338493212796775116102030799794358724497530713924646525414698841688697157985955712166061805209593917296689331549, 115903444433627393782263321545723293331734054164222225210892027584256572449507905186866785127399184849045448785046396243696852820257472478863680691688255061778989645008679725395796822001414659353031068139231954233716215642251222085345576111525329549764925342157273276401618189772044808880926946913777711318945, 160346862926570173155556405185673405256944925346630648110363303639494401757384328238794234360928308509443110543455974567280534799615104252084916968843795140271961900665652116021390071816425635325483939262810245627188737532006217565026586655932736202152482519271281991592202648637512700347912783277919685772993, 153756145373284063246387226074130484888653092242571108769579800148375684177761146700330667363028796981671449245310468559005581499287242595226649317375919855635375141187450874362896818902873290546399632649342407712407283763334294537805123192425374231179698233819636663246910878178808591809105825780945571002589, asn1_NOVALUE}, undefined,undefined,#Ref<0.0.0.73>, {<0.35.0>,#Ref<0.0.0.80>}, 0,<<>>,true} ** Reason for termination = ** {function_clause,[{pubkey_cert,select_extension,[{2,5,29,35},asn1_NOVALUE]}, {pubkey_cert,issuer_id,2}, {ssl_certificate,certificate_chain,4}, {ssl_handshake,certificate,3}, {ssl_connection,certify_server,1}, {ssl_connection,server_certify_and_key_exchange,1}, {ssl_connection,do_server_hello,2}, {lists,foldl,3}]} =CRASH REPORT==== 3-May-2010::15:57:29 === crasher: initial call: ssl_connection:init/1 pid: <0.60.0> registered_name: [] exception exit: {function_clause, [{pubkey_cert,select_extension, [{2,5,29,35},asn1_NOVALUE]}, {pubkey_cert,issuer_id,2}, {ssl_certificate,certificate_chain,4}, {ssl_handshake,certificate,3}, {ssl_connection,certify_server,1}, {ssl_connection,server_certify_and_key_exchange,1}, {ssl_connection,do_server_hello,2}, {lists,foldl,3}]} in function gen_fsm:terminate/7 ancestors: [ssl_connection_sup,ssl_sup,<0.50.0>] messages: [] links: [<0.54.0>] dictionary: [] trap_exit: false status: running heap_size: 610 stack_size: 24 reductions: 1822 neighbours: =SUPERVISOR REPORT==== 3-May-2010::15:57:29 === Supervisor: {local,ssl_connection_sup} Context: child_terminated Reason: {function_clause, [{pubkey_cert,select_extension, [{2,5,29,35},asn1_NOVALUE]}, {pubkey_cert,issuer_id,2}, {ssl_certificate,certificate_chain,4}, {ssl_handshake,certificate,3}, {ssl_connection,certify_server,1}, {ssl_connection,server_certify_and_key_exchange,1}, {ssl_connection,do_server_hello,2}, {lists,foldl,3}]} Offender: [{pid,<0.60.0>}, {name,undefined}, {mfa, {ssl_connection,start_link, [server,"localhost",10000,#Port<0.1154>, {{ssl_options,[],verify_none,#Fun, false,false,1, "TEST_CA/hosts/pacifico_server/cert.pem", "TEST_CA/hosts/pacifico_server/key.pem", undefined,[],"TEST_CA/ca-cert.pem", [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], #Fun,false,[]}, {socket_options,list,0,0,0,false}}, <0.35.0>, {gen_tcp,tcp,tcp_closed}]}}, {restart_type,temporary}, {shutdown,4000}, {child_type,worker}] Any ideas what might be going on? Regards, M From brady.mccary@REDACTED Mon May 3 21:26:17 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Mon, 3 May 2010 14:26:17 -0500 Subject: [erlang-questions] 'new' SSL API crash In-Reply-To: <20100503185312.GB75944@logik.internal.network> References: <20100503185312.GB75944@logik.internal.network> Message-ID: M, I think you have just disclosed a private encryption key on this list. Hopefully it is a dummy key. Brady On Mon, May 3, 2010 at 1:53 PM, wrote: > Hello. Is the 'ssl_new' module known to be broken, currently? > > I've tried using the 'old' ssl module but it seems to have no way to > actually fail SSL handshakes for non-verified connections and the like, > so I'm forced to use the new implementation. > > However, I can't get it to do anything without crashing. I know the > certificates are valid as I've tested them using the openssl s_client > and s_server command line utilities. > > Test program: > > -module (sslserv_new). > -export ([start/0]). > > server_ssl_settings () -> [ > ?%% Socket options. > ?{active, ? ? false}, > ?{reuseaddr, ?true}, > > ?%% SSL options. > ?{cacertfile, ? ? "TEST_CA/ca-cert.pem"}, > ?{certfile, ? ? ? "TEST_CA/hosts/pacifico_server/cert.pem"}, > ?{keyfile, ? ? ? ?"TEST_CA/hosts/pacifico_server/key.pem"}, > ?{ssl_imp, ? ? ? ?new}, > ?{verify, ? ? ? ? verify_peer}, > ?{depth, ? ? ? ? ?1}, > ?{ciphers, ? ? ? ?ssl:cipher_suites()}, > ?{reuse_sessions, false} > ]. > > start() -> > ?io:format ("ssl:listen\n"), > ?{ok, Socket} = ssl:listen (10000, server_ssl_settings ()), > > ?io:format ("ssl:transport_accept\n"), > ?{ok, Client_Socket} = ssl:transport_accept (Socket), > > ?io:format ("ssl:ssl_accept\n"), > ?case ssl:ssl_accept (Client_Socket) of > ? ?ok -> > ? ? ?io:format ("ssl:ssl_accept: accepted\n"), > ? ? ?io:format ("ssl:close\n"), > ? ? ?ok = ssl:close (Client_Socket); > ? ?{error, Reason} -> > ? ? ?io:format ("ssl:ssl_accept: error ~w\n", [Reason]) > ?end, > > ?io:format ("ssl:close\n"), > ?ok = ssl:close (Socket). > > -- > > Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] > Eshell V5.7.4 ?(abort with ^G) > > application:start (sasl), > application:start (crypto), > application:start (ssl), > > -- > > 1> ssl:versions(). > [{ssl_app,"3.10.7"}, > ?{supported,[tlsv1,sslv3]}, > ?{available,[tlsv1,sslv3]}] > 2> sslserv_new:start(). > ssl:listen > ssl:transport_accept > ssl:ssl_accept > > Then, connecting using 'openssl s_client' using a known, verifiable test certificate: > > ** exception exit: {{function_clause,[{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{lists,foldl,3}]}, > ? ? ? ? ? ? ? ? ? ?{gen_fsm,sync_send_all_state_event, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? [<0.60.0>,started,infinity]}} > ? ? in function ?gen_fsm:sync_send_all_state_event/3 > ? ? in call from ssl:ssl_accept/2 > ? ? in call from sslserv_new:start/0 > 3> > =ERROR REPORT==== 3-May-2010::15:57:29 === > ** State machine <0.60.0> terminating > ** Last event in was {ssl_tls,undefined,22, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{3,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?<<1,0,0,133,3,1,0,93,0,0,0,32,0,0,57,0,0,56,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?53,0,0,136,0,0,135,0,0,132,0,0,22,0,0,19,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?10,7,0,192,0,0,51,0,0,50,0,0,47,0,0,69,0,0,68, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,0,65,3,0,128,0,0,5,0,0,4,1,0,128,0,0,21,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?18,0,0,9,6,0,64,0,0,20,0,0,17,0,0,8,0,0,6,4,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?128,0,0,3,2,0,128,198,211,40,132,76,120,105, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?20,171,188,10,216,42,133,0,122,173,212,152, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?167,161,137,84,183,0,215,233,12,153,221,99,235>>} (for all states) > ** When State == hello > ** ? ? ?Data ?== {state,server, > ? ? ? ? ? ? ? ? ? ? {#Ref<0.0.0.70>,<0.35.0>}, > ? ? ? ? ? ? ? ? ? ? gen_tcp,tcp,tcp_closed,"localhost",10000,#Port<0.1154>, > ? ? ? ? ? ? ? ? ? ? {ssl_options,[],verify_none,#Fun,false, > ? ? ? ? ? ? ? ? ? ? ? ? false,1,"TEST_CA/hosts/pacifico_server/cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? "TEST_CA/hosts/pacifico_server/key.pem",undefined, > ? ? ? ? ? ? ? ? ? ? ? ? undefined,"TEST_CA/ca-cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > ? ? ? ? ? ? ? ? ? ? ? ? #Fun,false,[]}, > ? ? ? ? ? ? ? ? ? ? {socket_options,list,0,0,0,false}, > ? ? ? ? ? ? ? ? ? ? {connection_states, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,0,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0,undefined,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<75,222,242,105,194,191,18,141,171,244,247, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 203,234,237,111,11,152,119,181,103,91,155, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 92,85,84,17,57,121,20,164,73,110>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,0,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0,undefined,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<75,222,242,105,194,191,18,141,171,244,247, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 203,234,237,111,11,152,119,181,103,91,155, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 92,85,84,17,57,121,20,164,73,110>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}}, > ? ? ? ? ? ? ? ? ? ? <<>>,<<>>, > ? ? ? ? ? ? ? ? ? ? {{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, > ? ? ? ? ? ? ? ? ? ? ? ? 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0>>, > ? ? ? ? ? ? ? ? ? ? ? <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, > ? ? ? ? ? ? ? ? ? ? ? ? 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116,97, > ? ? ? ? ? ? ? ? ? ? ? ? 120,95,116,111,111,108,115,45,49,46,54,46,52,47,101, > ? ? ? ? ? ? ? ? ? ? ? ? 98,105,110,47,115,115,108,95,104,97,110,100,115,104, > ? ? ? ? ? ? ? ? ? ? ? ? 97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}, > ? ? ? ? ? ? ? ? ? ? ?{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, > ? ? ? ? ? ? ? ? ? ? ? ? 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0>>, > ? ? ? ? ? ? ? ? ? ? ? <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, > ? ? ? ? ? ? ? ? ? ? ? ? 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116, > ? ? ? ? ? ? ? ? ? ? ? ? 97,120,95,116,111,111,108,115,45,49,46,54,46,52,47, > ? ? ? ? ? ? ? ? ? ? ? ? 101,98,105,110,47,115,115,108,95,104,97,110,100,115, > ? ? ? ? ? ? ? ? ? ? ? ? 104,97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}}, > ? ? ? ? ? ? ? ? ? ? [], > ? ? ? ? ? ? ? ? ? ? <<48,130,3,93,48,130,2,69,2,1,1,48,13,6,9,42,134,72,134, > ? ? ? ? ? ? ? ? ? ? ? 247,13,1,1,5,5,0,48,129,136,49,17,48,15,6,3,85,4,10,19, > ? ? ? ? ? ? ? ? ? ? ? 8,80,65,67,73,70,73,67,79,49,20,48,18,6,3,85,4,11,20,11, > ? ? ? ? ? ? ? ? ? ? ? 112,97,99,105,102,105,99,111,95,99,97,49,36,48,34,6,9, > ? ? ? ? ? ? ? ? ? ? ? 42,134,72,134,247,13,1,9,1,22,21,112,97,99,105,102,105, > ? ? ? ? ? ? ? ? ? ? ? 99,111,95,99,97,64,108,111,99,97,108,104,111,115,116,49, > ? ? ? ? ? ? ? ? ? ? ? 10,48,8,6,3,85,4,7,19,1,46,49,10,48,8,6,3,85,4,8,19,1, > ? ? ? ? ? ? ? ? ? ? ? 46,49,11,48,9,6,3,85,4,6,19,2,90,90,49,18,48,16,6,3,85, > ? ? ? ? ? ? ? ? ? ? ? 4,3,19,9,108,111,99,97,108,104,111,115,116,48,30,23,13, > ? ? ? ? ? ? ? ? ? ? ? 49,48,48,53,48,50,49,51,52,57,52,51,90,23,13,50,48,48, > ? ? ? ? ? ? ? ? ? ? ? 52,50,57,49,51,52,57,52,51,90,48,96,49,11,48,9,6,3,85,4, > ? ? ? ? ? ? ? ? ? ? ? 6,19,2,90,90,49,10,48,8,6,3,85,4,8,19,1,46,49,17,48,15, > ? ? ? ? ? ? ? ? ? ? ? 6,3,85,4,10,19,8,80,65,67,73,70,73,67,79,49,24,48,22,6, > ? ? ? ? ? ? ? ? ? ? ? 3,85,4,11,20,15,112,97,99,105,102,105,99,111,95,115,101, > ? ? ? ? ? ? ? ? ? ? ? 114,118,101,114,49,24,48,22,6,3,85,4,3,20,15,112,97,99, > ? ? ? ? ? ? ? ? ? ? ? 105,102,105,99,111,95,115,101,114,118,101,114,48,130,1, > ? ? ? ? ? ? ? ? ? ? ? 34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,130,1,15, > ? ? ? ? ? ? ? ? ? ? ? 0,48,130,1,10,2,130,1,1,0,240,234,138,84,118,47,107,232, > ? ? ? ? ? ? ? ? ? ? ? 150,182,200,67,1,35,15,48,208,88,231,11,213,21,249,110, > ? ? ? ? ? ? ? ? ? ? ? 226,145,33,249,255,250,114,58,255,247,130,249,140,60, > ? ? ? ? ? ? ? ? ? ? ? 108,201,2,98,26,254,16,213,173,238,140,201,39,75,7,97, > ? ? ? ? ? ? ? ? ? ? ? 209,120,94,80,69,11,72,117,162,83,84,194,57,232,106,19, > ? ? ? ? ? ? ? ? ? ? ? 2,196,52,255,102,220,178,30,82,85,10,96,118,20,104,238, > ? ? ? ? ? ? ? ? ? ? ? 54,214,183,157,110,205,247,220,236,34,209,225,95,113, > ? ? ? ? ? ? ? ? ? ? ? 244,195,193,56,177,196,204,248,203,210,172,236,124,75, > ? ? ? ? ? ? ? ? ? ? ? 60,246,172,183,76,9,253,155,20,101,63,66,12,109,213,186, > ? ? ? ? ? ? ? ? ? ? ? 167,95,2,197,100,120,22,94,247,229,2,22,95,54,216,42,78, > ? ? ? ? ? ? ? ? ? ? ? 230,204,144,123,218,153,113,128,155,236,88,228,171,169, > ? ? ? ? ? ? ? ? ? ? ? 59,165,2,122,196,149,208,179,249,5,86,38,39,217,79,193, > ? ? ? ? ? ? ? ? ? ? ? 7,121,41,193,201,3,43,109,225,62,195,67,173,248,3,245, > ? ? ? ? ? ? ? ? ? ? ? 81,210,197,193,236,3,91,150,57,142,97,34,18,138,104,212, > ? ? ? ? ? ? ? ? ? ? ? 252,188,6,165,221,221,243,166,115,40,247,179,191,163, > ? ? ? ? ? ? ? ? ? ? ? 127,190,211,153,161,229,113,220,178,216,48,240,116,133, > ? ? ? ? ? ? ? ? ? ? ? 103,20,107,225,88,214,163,63,233,4,175,50,61,6,22,240, > ? ? ? ? ? ? ? ? ? ? ? 78,107,145,2,3,1,0,1,48,13,6,9,42,134,72,134,247,13,1,1, > ? ? ? ? ? ? ? ? ? ? ? 5,5,0,3,130,1,1,0,103,56,251,221,227,238,147,176,66,136, > ? ? ? ? ? ? ? ? ? ? ? 67,183,114,184,232,52,77,105,35,127,218,87,140,246,244, > ? ? ? ? ? ? ? ? ? ? ? 131,178,205,126,183,13,38,12,90,3,172,190,31,142,5,170, > ? ? ? ? ? ? ? ? ? ? ? 202,43,229,222,118,97,167,201,150,182,54,41,67,173,234, > ? ? ? ? ? ? ? ? ? ? ? 202,139,219,152,255,34,15,191,247,240,37,69,210,46,137, > ? ? ? ? ? ? ? ? ? ? ? 148,86,105,182,112,77,238,106,8,115,135,239,117,148,12, > ? ? ? ? ? ? ? ? ? ? ? 71,65,61,149,149,238,4,8,48,118,236,135,158,183,156,215, > ? ? ? ? ? ? ? ? ? ? ? 132,122,46,139,35,81,172,60,217,218,157,198,183,10,142, > ? ? ? ? ? ? ? ? ? ? ? 41,67,186,92,144,238,232,144,223,150,33,7,141,63,164, > ? ? ? ? ? ? ? ? ? ? ? 230,173,145,48,36,152,147,32,83,156,37,23,191,250,58,51, > ? ? ? ? ? ? ? ? ? ? ? 26,228,110,156,248,226,26,247,42,10,180,228,98,202,249, > ? ? ? ? ? ? ? ? ? ? ? 31,223,122,26,83,94,47,134,135,202,150,140,201,178,178, > ? ? ? ? ? ? ? ? ? ? ? 137,77,116,240,36,174,17,44,221,75,61,53,196,213,107,79, > ? ? ? ? ? ? ? ? ? ? ? 230,160,158,202,223,113,180,19,11,255,19,247,161,211, > ? ? ? ? ? ? ? ? ? ? ? 221,185,204,232,155,16,207,175,83,7,237,175,220,121,90, > ? ? ? ? ? ? ? ? ? ? ? 170,96,198,200,43,146,250,75,86,97,65,60,250,211,255, > ? ? ? ? ? ? ? ? ? ? ? 104,133,172,217,210,30,144,86,10,90,96,157,85,79,59,249, > ? ? ? ? ? ? ? ? ? ? ? 206,138>>, > ? ? ? ? ? ? ? ? ? ? {session,undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? undefined,false,undefined}, > ? ? ? ? ? ? ? ? ? ? 20499,ssl_session_cache,undefined,undefined,false, > ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? {'RSAPrivateKey','two-prime', > ? ? ? ? ? ? ? ? ? ? ? ? 30412849349635586059878763176680338042806370928303169072580788733024438138591624114344661613699569307282187776490585714522517050565529617950655734681469324828571314711428867434319920530980711392758700590981596083356704738486321722811445451684075618202284684847604912244382636109829288466866694913244559971697849183994297490539908862341588923396172960090070282527346558060132865021448687169667310568206513992904712186265982889729037153800223846493957161193620543020688239671219777230989849538520312806141702778165525520364118176466217430310054602386812099554335308017341261109213665732076271625040081260558423830719377, > ? ? ? ? ? ? ? ? ? ? ? ? 65537, > ? ? ? ? ? ? ? ? ? ? ? ? 15199231804150546259658042484791966551963578848813856243560714914822915640833292103958537953118116384964424618816792863691594075974988046092098009948916868274565455708125634299346634071611627633962886925804052924579735878963676936497600941132620179791254257637877896314147191365262961154085247394502328039931767245176290407517156245147167225384726350170243685109653662879990481102526168621741076751484642682168826381020813101510533444174790316691177259394914972358200408962095948054841259799747816506752270496315220170738960924479086469481940145231690103702610522190880399648129761477972116477505211965989493531309937, > ? ? ? ? ? ? ? ? ? ? ? ? 176191409302436409962613455746475864610360333729834662591534394409663736050830385559280211933901474704302550960882948381498437634097559237504384892632519298195621737913663090630551501287500290638791916696994910572811737440764017948768162498168387518624603640493626338734757174408297008712871342547046062968773, > ? ? ? ? ? ? ? ? ? ? ? ? 172612555118571442807091000766318626155131399021717029980434951225715253087609998682422104752170804119306391847675331885937326037489735337859546704781797012286523769446860097366850215779124291431114338493212796775116102030799794358724497530713924646525414698841688697157985955712166061805209593917296689331549, > ? ? ? ? ? ? ? ? ? ? ? ? 115903444433627393782263321545723293331734054164222225210892027584256572449507905186866785127399184849045448785046396243696852820257472478863680691688255061778989645008679725395796822001414659353031068139231954233716215642251222085345576111525329549764925342157273276401618189772044808880926946913777711318945, > ? ? ? ? ? ? ? ? ? ? ? ? 160346862926570173155556405185673405256944925346630648110363303639494401757384328238794234360928308509443110543455974567280534799615104252084916968843795140271961900665652116021390071816425635325483939262810245627188737532006217565026586655932736202152482519271281991592202648637512700347912783277919685772993, > ? ? ? ? ? ? ? ? ? ? ? ? 153756145373284063246387226074130484888653092242571108769579800148375684177761146700330667363028796981671449245310468559005581499287242595226649317375919855635375141187450874362896818902873290546399632649342407712407283763334294537805123192425374231179698233819636663246910878178808591809105825780945571002589, > ? ? ? ? ? ? ? ? ? ? ? ? asn1_NOVALUE}, > ? ? ? ? ? ? ? ? ? ? undefined,undefined,#Ref<0.0.0.73>, > ? ? ? ? ? ? ? ? ? ? {<0.35.0>,#Ref<0.0.0.80>}, > ? ? ? ? ? ? ? ? ? ? 0,<<>>,true} > ** Reason for termination = > ** {function_clause,[{pubkey_cert,select_extension,[{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? {pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? {ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? {ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? {lists,foldl,3}]} > > =CRASH REPORT==== 3-May-2010::15:57:29 === > ?crasher: > ? ?initial call: ssl_connection:init/1 > ? ?pid: <0.60.0> > ? ?registered_name: [] > ? ?exception exit: {function_clause, > ? ? ? ? ? ? ? ? ? ? ? ?[{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ? ? {pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {lists,foldl,3}]} > ? ? ?in function ?gen_fsm:terminate/7 > ? ?ancestors: [ssl_connection_sup,ssl_sup,<0.50.0>] > ? ?messages: [] > ? ?links: [<0.54.0>] > ? ?dictionary: [] > ? ?trap_exit: false > ? ?status: running > ? ?heap_size: 610 > ? ?stack_size: 24 > ? ?reductions: 1822 > ?neighbours: > > =SUPERVISOR REPORT==== 3-May-2010::15:57:29 === > ? ? Supervisor: {local,ssl_connection_sup} > ? ? Context: ? ?child_terminated > ? ? Reason: ? ? {function_clause, > ? ? ? ? ? ? ? ? ? ? [{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ?[{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ?{pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ?{lists,foldl,3}]} > ? ? Offender: ? [{pid,<0.60.0>}, > ? ? ? ? ? ? ? ? ?{name,undefined}, > ? ? ? ? ? ? ? ? ?{mfa, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,start_link, > ? ? ? ? ? ? ? ? ? ? ? ? ?[server,"localhost",10000,#Port<0.1154>, > ? ? ? ? ? ? ? ? ? ? ? ? ? {{ssl_options,[],verify_none,#Fun, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?false,false,1, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"TEST_CA/hosts/pacifico_server/cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"TEST_CA/hosts/pacifico_server/key.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?undefined,[],"TEST_CA/ca-cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?#Fun,false,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{socket_options,list,0,0,0,false}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? <0.35.0>, > ? ? ? ? ? ? ? ? ? ? ? ? ? {gen_tcp,tcp,tcp_closed}]}}, > ? ? ? ? ? ? ? ? ?{restart_type,temporary}, > ? ? ? ? ? ? ? ? ?{shutdown,4000}, > ? ? ? ? ? ? ? ? ?{child_type,worker}] > > Any ideas what might be going on? > > Regards, > M > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From prikrutil@REDACTED Mon May 3 21:57:19 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Mon, 3 May 2010 23:57:19 +0400 Subject: Special syntax for dictionaries Message-ID: Hello! While reading turorials of how MongoDB documents are made in Python/Ruby, I saw one thing I ever wanted to have in Erlang: {"author": "Mike", "text": "My first blog post!", "tags": ["mongodb", "python", "pymongo"], "date": datetime.datetime.utcnow()} That is special syntax for dictionaries. Take a look at the slide 47 of "What are the important ideas in Erlang?" [1] presentation by Joe Armstrong: -----Slide 47----- Hashmaps foo(<{a:X, b:Y | T }>) -> ... > foo(<{c:23, a:123, b:abc}>) Binds X=123, Y=abc T=<{c:23}> --------------------- Are there chances that we'll see this feature in one of the following releases of Erlang? I know there already are records, proplists and dict/gb_trees, but runtime support for records is limited and it isn't possible to pattern match on proplists (because the order of tuples may change) and dicts. [1] http://www.erlang-factory.com/conference/SFBay2010/speakers/joearmstrong -- Sergey Samokhin From org.erlang@REDACTED Mon May 3 22:44:14 2010 From: org.erlang@REDACTED (org.erlang@REDACTED) Date: Mon, 3 May 2010 20:44:14 +0000 Subject: [erlang-questions] 'new' SSL API crash In-Reply-To: References: <20100503185312.GB75944@logik.internal.network> Message-ID: <20100503204414.GC75944@logik.internal.network> On 2010-05-03 14:26:17, Brady McCary wrote: > M, > > I think you have just disclosed a private encryption key on this list. > Hopefully it is a dummy key. > > Brady Yeah, it's just a dummy. The CA's a dummy one too, just set up for testing. M From ok@REDACTED Mon May 3 23:55:41 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 4 May 2010 09:55:41 +1200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: <4BDEB93D.8070008@eonblast.com> References: <4BDD2F3B.2020605@eonblast.com> <4BDEB93D.8070008@eonblast.com> Message-ID: <24B60456-5B7D-4519-A3AF-DD39DD1AC846@cs.otago.ac.nz> On May 3, 2010, at 11:53 PM, Henning Diedrich wrote: > Thank you for the advice Richard! > > The function is to be called often and directly on non-tested data > coming in over the network. > > So at first sight, splitting up seems to bloat things instead of > making them clearer in this case. There is no moment in time, long > up front, when the diagnosis should be done. I'll give splitting up > a try to learn how that looks. > Funny, I thought the style I proposed was all about *removing* bloat. One of the reasons I got interested in literate programming, back when that was a new thing, was the fact that in typical imperative code, the normal case could easily get lost in code dedicated to checking arguments and reporting errors, and with literate programming I could at least reduce that to <> if (ok) { <> } else { <> } and then <> could be uncluttered. >> Why? What is the programmer supposed to do about it? >> More to the point, what is the *program* supposed to do about it? > 1) Discard the data he/it is trying to parse and tell the data > source that it was no good, and why. But to what level of detail. Recalling that the data is supposed to have the form {M,S,U} where M,S,U are all non-negative integers, "why" a term T might be no good could be - T is not a tuple - T is a tuple but not of arity 3 - T is a tuple of arity >= 1 whose first element is not a number - T is a tuple of arity >= 1 whose first element is not an integer - T is a tuple of arity >= 1 oh heck there are just TOO many possibilities. And the whole thing is ambiguous. Take {-1.2,fred,999999999999999999999}. Is it wrong - because the first argument ISN'T AN INTEGER - because the first argument IS NEGATIVE - because the second argument IS AN ATOM - because the third argument IS OUT OF RANGE (I don't think the code we've seen so far bothered to check that the arguments were within reasonable bounds, but it should have.) I've been through this once, when I designed the exception handling facility in Quintus Prolog and then had to go through all the source code ensuring that every single operation available to users reported reasonable errors. I'm going through it again in my Smalltalk system, gradually replacing "self error: 'I do not like thee Dr Fell'" with (NastyLecturerError subject: drFell critic: self) signal and the answer is that as soon as there is more than one value to inspect you *can't* report every possible error in a perfect way. Take something like anArray copyFrom: firstIndex to: lastIndex which does what it looks like (inclusive bounds). If lastIndex - firstIndex < -1, which one of them is at fault? Any guess you make might be wrong. I therefore respectfully suggest that any attempt to provide fine-grained diagnosis of "why" is misguided. To start with, if someone sets up a data source that is supposed to be sending you well formed time stamps, THEY MUST KNOW what a well formed time stamp is supposed to look like. You *can* tell them " is not a well formed timestamp". And that's *all* you can or should tell them. THEY have all the information they need to make a diagnosis in a way that makes sense for their situation. You do NOT. So here's what you should be doing. You have a protocol between you and your data source. Typically, you will have {Action,Operand1,...,Operandn}. Define types for the operands (in the UBF sense, which includes ranges). If an operand does not conform to its type, report that fact, AND NO MORE DETAIL THAN THAT. Better still, use UBF, and let UBF do the checking. You *can't* put the reporting in the function that decodes a timestamp, for the simple reason that it doesn't know who to report the error *to*, or how. The function that decodes a timestamp has that job and no other. It could possibly do decode({M,S,U}) when ... -> {ok,...}; decode(T) -> {error,invalid_timestamp,T}. But no more. > > 2) If the data was self-generated, e.g. for testing, the programmer > to use this function may make assumptions that are wrong and the > programmer to program this function (me) wants a way to inform him > in case he runs into trouble. WHY might the programmer make wrong assumptions? Is there something desperately wrong with your documentation? What is it about your situation that lets someone know "this operation needs a timestamp" but NOT know what a timestamp is supposed to look like? If they don't know what a timestamp is supposed to look like, how are they supposed to make sense of your detailed diagnostic. Above all, HOW ON EARTH are they supposed to make sense of a diagnostic that talks about which test in some guard failed, when they should never even see the guard in question? If you are testing, why can't you use UBF or a similar scheme of your own devising? > By providing more information than function_clause. Especially if I > can imagine what the wrong assumption may be. My absolutely uniform experience has been that when people imagine what the wrong assumption may be, they are WRONG. It is worse than useless to make such guesses. Don't ever waste anyone's time or bandwidth on imagined causes of errors, just report the errors themselves simply and clearly. You should not be reporting function_clause back to your data source in the first place. You should be reporting {type_error,timestamp,T} or something like that which is framed in terms of your PROTOCOL between your program and the data source, which neither reveals nor depends on ANY internal aspect of your program whatever. It may be that there is more about your program that you could tell us. From ok@REDACTED Tue May 4 01:10:09 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 4 May 2010 11:10:09 +1200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: On May 4, 2010, at 7:57 AM, Sergey Samokhin wrote: > Hello! > > While reading turorials of how MongoDB documents are made in > Python/Ruby, I saw one thing I ever wanted to have in Erlang: > > {"author": "Mike", > "text": "My first blog post!", > "tags": ["mongodb", "python", "pymongo"], > "date": datetime.datetime.utcnow()} > > That is special syntax for dictionaries. My proposal for 'frames' has been sitting around for years. I would write that as <{ author ~ "Mike" , text ~ "My first blog post!" , tags ~ ["mongodb","python","pymongo"] , date ~ erlang:now() }> but it's NOT a hashmap because it's NOT mutable, the keys may only be atoms, and it's intended for fast matching of groups of at most low dozens of entries, being a replacement for -record, not dictionaries. As a practical matter, it is important to separate keys from values by something other than : (module prefix), = (matching), or -> (function arrow), and a lot of work at Xerox PARC used ~ . Joe Armstrong has a similar proposal which he called 'proper structs' that is only superficially different from mine, but even older. (Mine discusses implementation, which his does not. It is possible to get the space for a frame down to the same as the space for the corresponding record in the usual case.) > From hd2010@REDACTED Tue May 4 04:40:29 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 May 2010 04:40:29 +0200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: <24B60456-5B7D-4519-A3AF-DD39DD1AC846@cs.otago.ac.nz> References: <4BDD2F3B.2020605@eonblast.com> <4BDEB93D.8070008@eonblast.com> <24B60456-5B7D-4519-A3AF-DD39DD1AC846@cs.otago.ac.nz> Message-ID: <4BDF891D.2050605@eonblast.com> Richard, thanks a lot for the detailed discussion, I follow almost all of it, here's where I am not sure. For context, the function is part of the encoding and decoding of data that is coming in from and going out to a VoltDB server. It will be used on a lower granular level to read out and create the contents of arrays, and SQL result table data. It is certain to at some point be crashed into invalid input due to errors of my programming of this API; or errors on part of the business logic programmer; or errors in the transmission if processing serialized data. There is potential for range errors, too, because {M,S,U} has a broader range than what VoltDB can store. Disallowing any negative values in M,S,U means for now disallowing timestamps before Christ. Theoretically, VoltDB could store timestamps ~4000 B.C. and at some point this should be made encompassed, and be as lenient as appropriate with negative values. The encoder function in question would receive time data from the domain logic into the API. It may be in any kind of shape if somebody did calculations with it. It may also be used to convert serialized data. 1 I do not produce the function_clause Error Reason myself, but this is what Erlang produces if I have the >= 0 checks in the guard. My original question was if there where means in the language maybe to enhance this message that the guards throw. At this point I come to believe that this would be a useful thing and actually the best solution if that would be possible. That would save the advantages of literate programming that you point out, with the least, well, bloat. Because if I can't, but still want to have a custom messages being thrown, I would have to pull it out of the guards, into the code, which is the first step to bloat, and give away the elegance of the guard and literate code structure. Or, as I understood from recent posts, split as below, which is another way of bloat, adding boilerplate in form of a function heads and repetitive guards. (And not quite the way of splitting diagnosis and job as you proposed). But despite the function head repetition, my understanding is growing that this is the way to do it in Erlang (I won't assume this to be important but I also come to like it): [untested] *wire_time({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro), Mega < 0, Sec < 0, Micro < 0 -> erlang:error(negative_input_value); wire_time({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro) -> MilliEpoch = Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000), % (*) <>.* --- Instead of: --- wire_time({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro) -> if (Mega >= 0), (Sec >= 0), (Micro >= 0) -> ok; true -> throw(negative_input_value) end, MilliEpoch = Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000), % (*) <>. --- Or: --- wire_time({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro), Mega >= 0, Sec >= 0, Micro >= 0 -> MilliEpoch = Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000), % (*) <>. However, once a better error detection is put into place, possibly accepting a mix of positive and negative values in the tuple, then it cannot be in the guards anymore in any event. It has to be calculated and then the result tested against the limits, instead of simply testing the tuple elements for >= 0. Would this then best be: *[untested] **-define(WIRE_MIN, ...). **-define(WIRE_MAX, ...). **wire_time({Mega, Sec, Micro}) when is_integer(Mega), is_integer(Sec), is_integer(Micro) -> wire_time(**milli_epoch({Mega, Sec, Micro})).* ** *wire_time(MilliEpoch) when MilliEpoch < ?WIRE_MIN -> erlang:error(time_underrun); **wire_time(MilliEpoch) when MilliEpoch > ?WIRE_MAX -> erlang:error(time_overrun); * *wire_time(MilliEpoch) when is_integer(MilliEpoch) -> <>; * * milli_epoch({Mega, Sec, Micro}) ->* * ** Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). * --- or, briefer variabled: --- *-define(WIRE_MIN, ...). **-define(WIRE_MAX, ...). **wire_time({M,S,U}=N) when is_integer(M), is_integer(S), is_integer(U) -> wire_time(milli_epoch(N)). wire_time(T) when T < ?WIRE_MIN -> erlang:error(time_underrun); wire_time(T) when T > ?WIRE_MAX -> erlang:error(time_overrun); wire_time(T) when is_integer(T) -> <>; milli_epoch({M,S,U}) -> M * ?BILLION + S * 1000 + trunc(U / 1000).* ** This would be split up then, though again not into (pre-)diagnosis and job. It shows a lot of boiler-repetitions of function heads, does it? 2 I maintain that in this case I am able to predict a wrong, naive interpretation of the error message. Or rather, that I can predict what will NOT come to mind as error source for the given Reason 'function_clause', until one inspects the source, or the docs; namely: that a part of the time tuple is negative. It's a less ambitious prediction than what you are proposing and defeating. But a programmer who is not me who is going to use this function, if she runs into the situation to have negative values fed into it, from getting 'function_clause' back, she will not know what to look for without looking at the implementation of the function and parsing the error message quite closely. Giving "negative_time_value" instead of "function_clause" will be a legitimate help to speed things up for her and not wrong. Trying to respond back to the server would mostly affect other functions, and not this one as I had erroneously proposed. I am willing to assume that even if I write a doc (and I will) it will not be read, or the remarks about the use of this function forgotten. I think such is life and it is forgivable. Giving a bit of a hint in errors can go a long way to boost GDP I think. Well, ok, reduce pain. Regards, Henning Richard O'Keefe wrote: > > On May 3, 2010, at 11:53 PM, Henning Diedrich wrote: > >> Thank you for the advice Richard! >> >> The function is to be called often and directly on non-tested data >> coming in over the network. >> >> So at first sight, splitting up seems to bloat things instead of >> making them clearer in this case. There is no moment in time, long up >> front, when the diagnosis should be done. I'll give splitting up a >> try to learn how that looks. >> > > Funny, I thought the style I proposed was all about *removing* bloat. > > One of the reasons I got interested in literate programming, back when > that was a new thing, was the fact that in typical imperative code, > the normal case could easily get lost in code dedicated to checking > arguments and reporting errors, and with literate programming I could > at least reduce that to > <> > if (ok) { > <> > } else { > <> > } > and then <> could be uncluttered. > >>> Why? What is the programmer supposed to do about it? >>> More to the point, what is the *program* supposed to do about it? >> 1) Discard the data he/it is trying to parse and tell the data source >> that it was no good, and why. > > But to what level of detail. > Recalling that the data is supposed to have the form > {M,S,U} where M,S,U are all non-negative integers, > "why" a term T might be no good could be > > - T is not a tuple > - T is a tuple but not of arity 3 > - T is a tuple of arity >= 1 > whose first element is not a number > - T is a tuple of arity >= 1 > whose first element is not an integer > - T is a tuple of arity >= 1 > oh heck there are just TOO many possibilities. > > And the whole thing is ambiguous. > Take {-1.2,fred,999999999999999999999}. > Is it wrong > - because the first argument ISN'T AN INTEGER > - because the first argument IS NEGATIVE > - because the second argument IS AN ATOM > - because the third argument IS OUT OF RANGE > (I don't think the code we've seen so far bothered to check that > the arguments were within reasonable bounds, but it should have.) > > I've been through this once, when I designed the exception handling > facility in Quintus Prolog and then had to go through all the source > code ensuring that every single operation available to users > reported reasonable errors. I'm going through it again in my > Smalltalk system, gradually replacing > "self error: 'I do not like thee Dr Fell'" > with > (NastyLecturerError subject: drFell critic: self) signal > and the answer is that as soon as there is more than one value to > inspect you *can't* report every possible error in a perfect way. > Take something like > anArray copyFrom: firstIndex to: lastIndex > which does what it looks like (inclusive bounds). > If lastIndex - firstIndex < -1, which one of them is at fault? > Any guess you make might be wrong. > > I therefore respectfully suggest that any attempt to provide > fine-grained diagnosis of "why" is misguided. To start with, > if someone sets up a data source that is supposed to be sending > you well formed time stamps, THEY MUST KNOW what a well formed > time stamp is supposed to look like. You *can* tell them > " is not a well formed timestamp". And that's *all* you > can or should tell them. THEY have all the information they > need to make a diagnosis in a way that makes sense for their > situation. You do NOT. > > So here's what you should be doing. > > You have a protocol between you and your data source. > Typically, you will have {Action,Operand1,...,Operandn}. > Define types for the operands (in the UBF sense, which > includes ranges). If an operand does not conform to its > type, report that fact, AND NO MORE DETAIL THAN THAT. > > Better still, use UBF, and let UBF do the checking. > > You *can't* put the reporting in the function that > decodes a timestamp, for the simple reason that it doesn't > know who to report the error *to*, or how. The function > that decodes a timestamp has that job and no other. > It could possibly do > > decode({M,S,U}) when ... -> {ok,...}; > decode(T) -> {error,invalid_timestamp,T}. > > But no more. > >> >> 2) If the data was self-generated, e.g. for testing, the programmer >> to use this function may make assumptions that are wrong and the >> programmer to program this function (me) wants a way to inform him in >> case he runs into trouble. > > WHY might the programmer make wrong assumptions? > Is there something desperately wrong with your documentation? > > What is it about your situation that lets someone know "this > operation needs a timestamp" but NOT know what a timestamp > is supposed to look like? If they don't know what a timestamp > is supposed to look like, how are they supposed to make sense > of your detailed diagnostic. Above all, HOW ON EARTH are they > supposed to make sense of a diagnostic that talks about which > test in some guard failed, when they should never even see the > guard in question? > > If you are testing, why can't you use UBF or a similar scheme of > your own devising? > >> By providing more information than function_clause. Especially if I >> can imagine what the wrong assumption may be. > > My absolutely uniform experience has been that when people imagine what > the wrong assumption may be, they are WRONG. It is worse than useless > to make such guesses. Don't ever waste anyone's time or bandwidth on > imagined causes of errors, just report the errors themselves simply and > clearly. > > You should not be reporting function_clause back to your data source in > the first place. You should be reporting {type_error,timestamp,T} or > something like that which is framed in terms of your PROTOCOL between > your program and the data source, which neither reveals nor depends on > ANY internal aspect of your program whatever. > > It may be that there is more about your program that you could tell us. > > From hd2010@REDACTED Tue May 4 04:47:50 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 May 2010 04:47:50 +0200 Subject: [erlang-questions] Debug support on for guards? In-Reply-To: <4BDF891D.2050605@eonblast.com> References: <4BDD2F3B.2020605@eonblast.com> <4BDEB93D.8070008@eonblast.com> <24B60456-5B7D-4519-A3AF-DD39DD1AC846@cs.otago.ac.nz> <4BDF891D.2050605@eonblast.com> Message-ID: <4BDF8AD6.5030904@eonblast.com> Mega < 0, Sec < 0, Micro < 0 is wrong of course. It should be or. Henning Diedrich wrote: > Richard, > > thanks a lot for the detailed discussion, I follow almost all of it, > here's where I am not sure. > > For context, the function is part of the encoding and decoding of data > that is coming in from and going out to a VoltDB server. It will be > used on a lower granular level to read out and create the contents of > arrays, and SQL result table data. It is certain to at some point be > crashed into invalid input due to errors of my programming of this > API; or errors on part of the business logic programmer; or errors in > the transmission if processing serialized data. There is potential for > range errors, too, because {M,S,U} has a broader range than what > VoltDB can store. Disallowing any negative values in M,S,U means for > now disallowing timestamps before Christ. Theoretically, VoltDB could > store timestamps ~4000 B.C. and at some point this should be made > encompassed, and be as lenient as appropriate with negative values. > > The encoder function in question would receive time data from the > domain logic into the API. It may be in any kind of shape if somebody > did calculations with it. It may also be used to convert serialized data. > > 1 > > I do not produce the function_clause Error Reason myself, but this is > what Erlang produces if I have the >= 0 checks in the guard. > > My original question was if there where means in the language maybe to > enhance this message that the guards throw. > > At this point I come to believe that this would be a useful thing and > actually the best solution if that would be possible. That would save > the advantages of literate programming that you point out, with the > least, well, bloat. > > Because if I can't, but still want to have a custom messages being > thrown, I would have to pull it out of the guards, into the code, > which is the first step to bloat, and give away the elegance of the > guard and literate code structure. > > Or, as I understood from recent posts, split as below, which is > another way of bloat, adding boilerplate in form of a function heads > and repetitive guards. (And not quite the way of splitting diagnosis > and job as you proposed). > > But despite the function head repetition, my understanding is growing > that this is the way to do it in Erlang (I won't assume this to be > important but I also come to like it): > > [untested] > > *wire_time({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro), > Mega < 0, Sec < 0, Micro < 0 -> > erlang:error(negative_input_value); > > wire_time({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro) -> > > MilliEpoch = Mega * ?BILLION + Sec * 1000 + trunc(Micro / > 1000), % (*) > <>.* > > > > --- Instead of: --- > > wire_time({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro) -> > > if (Mega >= 0), (Sec >= 0), (Micro >= 0) -> ok; > true -> throw(negative_input_value) end, > MilliEpoch = Mega * ?BILLION + Sec * 1000 + > trunc(Micro / > 1000), % (*) > <>. > > --- Or: --- > > wire_time({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro), > Mega >= 0, Sec >= 0, Micro >= 0 -> > MilliEpoch = Mega * ?BILLION + Sec * 1000 + > trunc(Micro / > 1000), % (*) > <>. > > However, once a better error detection is put into place, possibly > accepting a mix of positive and negative values in the tuple, then it > cannot be in the guards anymore in any event. It has to be calculated > and then the result tested against the limits, instead of simply > testing the tuple elements for >= 0. > > Would this then best be: > > *[untested] > > **-define(WIRE_MIN, ...). > **-define(WIRE_MAX, ...). > > **wire_time({Mega, Sec, Micro}) when is_integer(Mega), > is_integer(Sec), is_integer(Micro) -> > wire_time(**milli_epoch({Mega, Sec, Micro})).* > ** > *wire_time(MilliEpoch) when MilliEpoch < ?WIRE_MIN -> > erlang:error(time_underrun); > > **wire_time(MilliEpoch) when MilliEpoch > ?WIRE_MAX -> > erlang:error(time_overrun); > * > *wire_time(MilliEpoch) when is_integer(MilliEpoch) -> > <>; > * > * milli_epoch({Mega, Sec, Micro}) ->* > * ** Mega * ?BILLION + Sec * 1000 + trunc(Micro / 1000). > * > > --- or, briefer variabled: --- > > *-define(WIRE_MIN, ...). > **-define(WIRE_MAX, ...). > > **wire_time({M,S,U}=N) when is_integer(M), is_integer(S), > is_integer(U) -> > wire_time(milli_epoch(N)). > > wire_time(T) when T < ?WIRE_MIN -> > erlang:error(time_underrun); > > wire_time(T) when T > ?WIRE_MAX -> > erlang:error(time_overrun); > > wire_time(T) when is_integer(T) -> > <>; > > milli_epoch({M,S,U}) -> > M * ?BILLION + S * 1000 + trunc(U / 1000).* > ** > > This would be split up then, though again not into (pre-)diagnosis and > job. It shows a lot of boiler-repetitions of function heads, does it? > > 2 > > I maintain that in this case I am able to predict a wrong, naive > interpretation of the error message. Or rather, that I can predict > what will NOT come to mind as error source for the given Reason > 'function_clause', until one inspects the source, or the docs; namely: > that a part of the time tuple is negative. > > It's a less ambitious prediction than what you are proposing and > defeating. > > But a programmer who is not me who is going to use this function, if > she runs into the situation to have negative values fed into it, from > getting 'function_clause' back, she will not know what to look for > without looking at the implementation of the function and parsing the > error message quite closely. Giving "negative_time_value" instead of > "function_clause" will be a legitimate help to speed things up for her > and not wrong. > > Trying to respond back to the server would mostly affect other > functions, and not this one as I had erroneously proposed. > > I am willing to assume that even if I write a doc (and I will) it will > not be read, or the remarks about the use of this function forgotten. > I think such is life and it is forgivable. Giving a bit of a hint in > errors can go a long way to boost GDP I think. Well, ok, reduce pain. > > Regards, > Henning > > > > Richard O'Keefe wrote: >> >> On May 3, 2010, at 11:53 PM, Henning Diedrich wrote: >> >>> Thank you for the advice Richard! >>> >>> The function is to be called often and directly on non-tested data >>> coming in over the network. >>> >>> So at first sight, splitting up seems to bloat things instead of >>> making them clearer in this case. There is no moment in time, long >>> up front, when the diagnosis should be done. I'll give splitting up >>> a try to learn how that looks. >>> >> >> Funny, I thought the style I proposed was all about *removing* bloat. >> >> One of the reasons I got interested in literate programming, back when >> that was a new thing, was the fact that in typical imperative code, >> the normal case could easily get lost in code dedicated to checking >> arguments and reporting errors, and with literate programming I could >> at least reduce that to >> <> >> if (ok) { >> <> >> } else { >> <> >> } >> and then <> could be uncluttered. >> >>>> Why? What is the programmer supposed to do about it? >>>> More to the point, what is the *program* supposed to do about it? >>> 1) Discard the data he/it is trying to parse and tell the data >>> source that it was no good, and why. >> >> But to what level of detail. >> Recalling that the data is supposed to have the form >> {M,S,U} where M,S,U are all non-negative integers, >> "why" a term T might be no good could be >> >> - T is not a tuple >> - T is a tuple but not of arity 3 >> - T is a tuple of arity >= 1 >> whose first element is not a number >> - T is a tuple of arity >= 1 >> whose first element is not an integer >> - T is a tuple of arity >= 1 >> oh heck there are just TOO many possibilities. >> >> And the whole thing is ambiguous. >> Take {-1.2,fred,999999999999999999999}. >> Is it wrong >> - because the first argument ISN'T AN INTEGER >> - because the first argument IS NEGATIVE >> - because the second argument IS AN ATOM >> - because the third argument IS OUT OF RANGE >> (I don't think the code we've seen so far bothered to check that >> the arguments were within reasonable bounds, but it should have.) >> >> I've been through this once, when I designed the exception handling >> facility in Quintus Prolog and then had to go through all the source >> code ensuring that every single operation available to users >> reported reasonable errors. I'm going through it again in my >> Smalltalk system, gradually replacing >> "self error: 'I do not like thee Dr Fell'" >> with >> (NastyLecturerError subject: drFell critic: self) signal >> and the answer is that as soon as there is more than one value to >> inspect you *can't* report every possible error in a perfect way. >> Take something like >> anArray copyFrom: firstIndex to: lastIndex >> which does what it looks like (inclusive bounds). >> If lastIndex - firstIndex < -1, which one of them is at fault? >> Any guess you make might be wrong. >> >> I therefore respectfully suggest that any attempt to provide >> fine-grained diagnosis of "why" is misguided. To start with, >> if someone sets up a data source that is supposed to be sending >> you well formed time stamps, THEY MUST KNOW what a well formed >> time stamp is supposed to look like. You *can* tell them >> " is not a well formed timestamp". And that's *all* you >> can or should tell them. THEY have all the information they >> need to make a diagnosis in a way that makes sense for their >> situation. You do NOT. >> >> So here's what you should be doing. >> >> You have a protocol between you and your data source. >> Typically, you will have {Action,Operand1,...,Operandn}. >> Define types for the operands (in the UBF sense, which >> includes ranges). If an operand does not conform to its >> type, report that fact, AND NO MORE DETAIL THAN THAT. >> >> Better still, use UBF, and let UBF do the checking. >> >> You *can't* put the reporting in the function that >> decodes a timestamp, for the simple reason that it doesn't >> know who to report the error *to*, or how. The function >> that decodes a timestamp has that job and no other. >> It could possibly do >> >> decode({M,S,U}) when ... -> {ok,...}; >> decode(T) -> {error,invalid_timestamp,T}. >> >> But no more. >> >>> >>> 2) If the data was self-generated, e.g. for testing, the programmer >>> to use this function may make assumptions that are wrong and the >>> programmer to program this function (me) wants a way to inform him >>> in case he runs into trouble. >> >> WHY might the programmer make wrong assumptions? >> Is there something desperately wrong with your documentation? >> >> What is it about your situation that lets someone know "this >> operation needs a timestamp" but NOT know what a timestamp >> is supposed to look like? If they don't know what a timestamp >> is supposed to look like, how are they supposed to make sense >> of your detailed diagnostic. Above all, HOW ON EARTH are they >> supposed to make sense of a diagnostic that talks about which >> test in some guard failed, when they should never even see the >> guard in question? >> >> If you are testing, why can't you use UBF or a similar scheme of >> your own devising? >> >>> By providing more information than function_clause. Especially if I >>> can imagine what the wrong assumption may be. >> >> My absolutely uniform experience has been that when people imagine what >> the wrong assumption may be, they are WRONG. It is worse than useless >> to make such guesses. Don't ever waste anyone's time or bandwidth on >> imagined causes of errors, just report the errors themselves simply and >> clearly. >> >> You should not be reporting function_clause back to your data source in >> the first place. You should be reporting {type_error,timestamp,T} or >> something like that which is framed in terms of your PROTOCOL between >> your program and the data source, which neither reveals nor depends on >> ANY internal aspect of your program whatever. >> >> It may be that there is more about your program that you could tell us. >> >> > From bernie@REDACTED Tue May 4 06:23:22 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 04 May 2010 14:23:22 +1000 Subject: Designing supervision trees Message-ID: <4BDFA13A.4090408@m5net.com> Hi list, I'm in the process of going through the design of the supervision tree for our application and it's rapidly becoming obvious to me that I could be a lot clearer on how supervision trees are "meant" to be structured to make a system as fault-tolerant as possible. Let me ask a couple of concrete questions and maybe it will help: * Is it kosher to have a supervised process (say a gen_server) start its own helper process(es) simply using spawn_link()? It seems like it should be fine - any failure of either will propagate over the link, causing them both to be shut down and the supervisor will then restart the main one which in turn will restart the helper. I say it "seems like it should be fine", but after reading all the supervisor and OTP docs I could lay my hands on I'm not really sure if there isn't some good reason to avoid this arrangement. * Let's say I have two processes, A and B. The state of B is dependent on the messages it has received from A. The particular example I'm dealing with is a process (B) who is responsible for starting and stopping apps and another (A) which is responsible for synchronising data with a remote store. The apps should be started when we are synced and stopped if we lose connection with the store. I don't necessarily want to merge them in to one process because A needs to be relatively responsive to the remote store, but the process of starting/stopping apps can take some time. There may be a much better way to arrange this, but I'm not exactly sure what it is... So we're up and running, remote store is synced, all the apps are running, and B crashes. I'm trying to figure out the "right" way to manage recovery - possibilities I can think of: - Have A and B under a one-for-all supervisor so that we just nuke the broader state and start it all again (seems like we should be able to recover with less impact than this). - Have B's state stored in an ETS table owned by its parent so that it can recover into it's previous state (that seems far too much like global data to me). - Have B query A for the current state on startup - that would work, except that it leaves us with multiple communication methods between A and B, one where B asks for the state and one where A pushes state updates - that seems a little redundant (and like extra code to maintain which nobody wants). - Get rid of B entirely and perhaps have A spawn a temporary processes to do app start/stop when required - seems a little messy, and prone to race conditions if we do multiple start/stop operations in quick succession... Any thoughts (and pointers to docs that deal with this stuff) are appreciated. As always, thanks if you read this far :) Cheers, Bernard From dougedmunds@REDACTED Tue May 4 06:52:28 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Mon, 03 May 2010 21:52:28 -0700 Subject: Process unexpectedly locks up Message-ID: <4BDFA80C.8010809@gmail.com> Hello, I'm posting a module (conn3.erl) below. This module builds a hierarchical tree of PIDs. There are two loops, one for entries, and another for the position in the tree(called 'me'). Each 'entry' runs a copy of the entry_loop. Each entry keeps track of its parent (one pid) and its children (list of 0 or more pids). Entries are not called directly. The me process runs in the me_loop. It manages the entries, and moves up and down in the tree, via: me ! add, me ! up, me ! down, me ! delete, me ! show. The bug I've encountered is when trying to move down the tree when there are multiple children. Here's the basic scenario: After running conn3:start(), type: me ! add. me ! add. to create two children. Now type: me ! down. Because there are more than one child, the code calls indexlist/3, which returns a list of tuples: [{1, PID1}, {2, PID2}, ...}. Then the next line in the 'down' message prints that list. After that the user is supposed to pick the child by using the integer: Input = get_user_input("Enter key: "), But the pid never gets to that line. I get no error message. It fails both in Windows XP and in a linux os. If someone can figure out how to allow user input of a selection, much appreciated. Doug Edmunds --------------------------- I've received this reply from Sam Bobroff, which indicates there is some kind of conflict with "io:wait_io_mon_reply". [Sam wrote:] Getting a backtrace often helps. This is what I did: $ erl Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> conn3_full:start(). - --setting me to top: self(): <0.34.0> M: <0.35.0> K: [] P: [] - --setting top self(): <0.35.0> M: <0.35.0> K: [] P [] ok - --show self():<0.34.0> M: <0.35.0> K: [<0.37.0>,<0.36.0>] P: [] 2> me ! down. [{1,<0.37.0>},{2,<0.36.0>}] down 3> {backtrace, BT} = process_info(whereis(me), backtrace). {backtrace,<<"Program counter: 0x0079f3c8 (io:wait_io_mon_reply/2 + 28)\nCP: 0x00000000 (invalid)\narity = 0\n\n0x002f6cbc Ret"...>>} 4> io:fwrite("~s\n", [binary_to_list(BT)]). Program counter: 0x0079f3c8 (io:wait_io_mon_reply/2 + 28) CP: 0x00000000 (invalid) arity = 0 0x002f6cbc Return addr 0x007a14c0 (conn3_full:get_user_input/1 + 20) y(0) #Ref<0.0.0.37> y(1) <0.25.0> 0x002f6cc8 Return addr 0x007a0bf4 (conn3_full:me_loop/3 + 676) 0x002f6ccc Return addr 0x001a1df4 () y(0) [] y(1) [{1,<0.37.0>},{2,<0.36.0>}] y(2) [] y(3) [<0.37.0>,<0.36.0>] y(4) <0.35.0> ok I can see that "me" is still in it's loop and that it's currently in "io:wait_io_mon_reply". I don't know exactly what this function is but my guess would be it's something to do with the shell and io:get_line (actually wait_io_mon_reply) fighting over the terminal input. If we try again with -noshell it might be better but then we won't be able to use the shell to send messages to "me". So, I modified the source to add "me ! down" in the set up sequence at line 16, and also uncommented the debug at the top of me_loop, and now I get: $ erl -noshell -run conn3_full - --me_loop self(): <0.29.0> M: [] K:[] P: [] - --setting top self(): <0.30.0> M: <0.30.0> K: [] P [] - --setting me to top: self(): <0.29.0> M: <0.30.0> K: [] P: [] - --me_loop self(): <0.29.0> M: <0.30.0> K:[] P: [] - --me_loop self(): <0.29.0> M: <0.30.0> K:[<0.31.0>] P: [] - --me_loop self(): <0.29.0> M: <0.30.0> K:[<0.32.0>,<0.31.0>] P: [] - --show self():<0.29.0> M: <0.30.0> K: [<0.32.0>,<0.31.0>] P: [] - --me_loop self(): <0.29.0> M: <0.30.0> K:[<0.32.0>,<0.31.0>] P: [] [{1,<0.32.0>},{2,<0.31.0>}] Enter key: 2 - --me_loop self(): <0.29.0> M: <0.30.0> K:[<0.32.0>,<0.31.0>] P: [] - --me_loop self(): <0.29.0> M: <0.31.0> K:[] P: <0.30.0> I entered "2" at the prompt and the loop has continued :-) --- End ------------------------ -module(conn3_full). -compile(export_all). %% usage conn:start(). %% then send messages to 'me' (see me_loop) start() -> % process_flag(trap_exit, true), Me = spawn(?MODULE, me_loop,[[],[],[]]), register(me, Me), Top = spawn(?MODULE, entry_loop,[[],[],[]]), register(top, Top), me ! {first_time}, top ! {first_time}, % uncomment this next line to get to the problem faster % me ! add, me ! add, me! show, ok. me_loop(M,K,P) -> % io:format("--me_loop self(): ~p M: ~p K:~p P: ~p~n",[self(),M,K,P]), receive {first_time} -> NM = whereis(top), io:format("--setting me to top: self(): ~p M: ~p K: ~p P: ~p ~n", [self(), whereis(top), K, P]), NM = whereis(top), NK = K, NP = P; show -> io:format("--show self():~p M: ~p K: ~p P: ~p~n",[self(),M,K,P]), NM = M, NK = K, NP = P; add -> %%create an entry Pid = spawn(?MODULE,entry_loop,[[],[],M]), Pid ! {set_pid, Pid}, %%update the entry that 'me' is copying M ! {p_add_kid, Pid}, %%update 'me' K2 = [Pid|K], NM = M, NK = K2, NP = P; del -> case P of [] -> io:format("--At the top~n"); _ -> P ! {p_update_kids, M, K}, ok = connect_kids_to_P(K,P), M ! die, me ! up end, NM = M, NK = K, NP = P; down -> case length(K) of 0 -> io:format("--No kids~n"); 1 -> [Head |_] = K, Head ! {self(), info_request}; _ -> Out = indexlist(1, K, []), ok = io:format("~p~n", [Out]), %%%%%% When more than one 'kid', %%%%%% process drops out of loop here. BUG? Input = get_user_input("Enter key: "), {Int,Rest} = string:to_integer(Input), case is_integer(Int) andalso Rest == [] of true -> Pick = pick_pid(Out,Int), case is_pid(Pick) of true -> Pick ! {self(), info_request}; _ -> io:format("that number is not on the list~n") end; _ -> io:format("must enter an integer~n") end end, NM = M, NK = K, NP = P; up -> case P of [] -> io:format("--At the top~n"); _ -> P ! {self(),info_request} end, NM = M, NK = K, NP = P; {info_requested, M2, K2, P2} -> NM = M2, NK = K2, NP = P2; die -> exit("killed"), io:format("~p died~n", [self()]), NM = M, NK = K, NP = P; Anything -> io:format("--me_loop got this:~p~n", [Anything]), NM = M, NK = K, NP = P end, me_loop (NM,NK,NP). entry_loop(M,K,P) -> % io:format("--entry_loop self(): ~p M: ~p K:~p P: ~p~n",[self(),M,K,P]), receive {first_time} -> io:format("--setting top self(): ~p M: ~p K: ~p P ~p ~n", [self(), whereis(top), K, P]), NM = whereis(top), NK = K, NP = P; show -> io:format("--show self():~p M: ~p K: ~p P: ~p~n",[self(),M,K,P]), NM = M, NK = K, NP = P; {set_pid, Pid} -> NM = Pid, NK = K, NP = P; {From,info_request} -> From ! {info_requested, M, K, P}, NM = M, NK = K, NP = P; {p_update_kids, Kid, GrandKidsList} -> K2 = lists:delete(Kid, K), K3 = lists:append(GrandKidsList,K2), %%still have to move me NM = M, NK = K3, NP = P; {kid_change_p, GrandP} -> P2 = GrandP, NM = M, NK = K, NP = P2; {p_add_kid, Pid} -> K2 = [Pid|K], NM = M, NK = K2, NP = P; % {tell_kids_about_Pid, Pid, Msg} -> % Pidlist = [Pidx || Pidx <- K, is_pid(Pid), Pid /= Pidx], % %%% exclude Pid % %% io:format("--Pid list: ~p~n",[Pidlist]), % ok = tell_list(Pidlist, Pid, Msg), % NM = M, NK = K, NP = P; die -> exit("killed"), io:format("~p died~n", [self()]), NM = M, NK = K, NP = P; Anything -> io:format("--entry_loop Got this:~p~n", [Anything]), NM = M, NK = K, NP = P end, %% io:format("here i am~n"), entry_loop (NM,NK,NP). indexlist(Start, [H|T],Out) -> NewOut = lists: append ([{Start, H}], Out), Start2 = Start+1, indexlist(Start2, T, NewOut); indexlist(_, [], Out) -> lists:reverse(Out). pick_pid(Out, Key) -> NewDict = dict:from_list(Out), case dict:is_key(Key,NewDict) of true -> dict:fetch(Key,NewDict); false -> "no such key" end. get_user_input( Prompt ) -> string:strip( % remove spaces from front and back string:strip( % remove line-feed from the end io:get_line( Prompt), right, $\n)). connect_kids_to_P([],_) -> ok; connect_kids_to_P(K,P) -> [H|T] = K, H ! {kid_change_p,P}, connect_kids_to_P(T,P). %%%not implemented % tell_list([],_,_) -> ok; % tell_list([H|T],X,Msg) -> H ! {Msg, X}, tell_list(T,X, Msg). %%%macro-ish utility b_alive(String) -> % ie b_alive("<0.35.0>") is_process_alive(list_to_pid(String)). From sam@REDACTED Tue May 4 07:41:43 2010 From: sam@REDACTED (Sam Bobroff) Date: Tue, 04 May 2010 15:41:43 +1000 Subject: [erlang-questions] Process unexpectedly locks up In-Reply-To: <4BDFA80C.8010809@gmail.com> References: <4BDFA80C.8010809@gmail.com> Message-ID: <4BDFB397.4040206@m5net.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Doug (and the list), (I'll reply to the questions list) [...] > I've received this reply from Sam Bobroff, which indicates > there is some kind of conflict with "io:wait_io_mon_reply". I'll add to my previous reply: I think you have two options: 1. Use the interactive shell (don't use "-noshell" on the command line): Since you can't use io:get_line(), you could modify the "down" handler in me_loop() in two places: * if the down case requires a decision, just print out the options (as now) and otherwise don't do anything. * add a handler for a {down, X} message that takes choice X as if it had been returned from get_user_input() in the old code. So now you do "me ! down" and get a message that it failed, with your options, then you do "me ! {down, 1}" to pick one. You never use get_line() so there is no conflict. 2. Don't use the shell (so *do* use "-noinput"): Since you're not using the shell, you can't drive the threads using messages. Instead, have the start() function continue on into an interactive loop calling get_line() and translating it into up, down or whatever, messages sent to "me". Then when you want to make a choice you can just call get_user_input() as you do now. I don't really think there's a bug (in Erlang) but the documentation for get_line would certainly be better if it said something about the interaction between the shell and the io input functions. Cheers, Sam. - -- Sam Bobroff | sam@REDACTED | M5 Networks Why does my email have those funny headers? Because I use PGP to sign my email (and you should too!): that's how you know it's really from me. See: http://en.wikipedia.org/wiki/Pretty_Good_Privacy -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvfs5YACgkQm97/UHSa/AQg3QCfT/Gmk+TQujmgv2NE+lMCAICB GvAAn3ICuTCwetbNz4zFQEBpUDAKp0La =rJh5 -----END PGP SIGNATURE----- From ingela@REDACTED Tue May 4 11:46:15 2010 From: ingela@REDACTED (Ingela Andin) Date: Tue, 4 May 2010 11:46:15 +0200 Subject: [erlang-questions] 'new' SSL API crash In-Reply-To: <20100503185312.GB75944@logik.internal.network> References: <20100503185312.GB75944@logik.internal.network> Message-ID: Hi! It seems like the select_extension function is missing a clause that handles certificates that does not have any extensions at all. I will fix this in the upcoming release. Would you mind sharing your dummy certs and key with the otp team for testing purposes? You can send them to me off the list if that is ok with you. Regards Ingela - Erlang/OTP team Ericsson AB 2010/5/3 : > Hello. Is the 'ssl_new' module known to be broken, currently? > > I've tried using the 'old' ssl module but it seems to have no way to > actually fail SSL handshakes for non-verified connections and the like, > so I'm forced to use the new implementation. > > However, I can't get it to do anything without crashing. I know the > certificates are valid as I've tested them using the openssl s_client > and s_server command line utilities. > > Test program: > > -module (sslserv_new). > -export ([start/0]). > > server_ssl_settings () -> [ > ?%% Socket options. > ?{active, ? ? false}, > ?{reuseaddr, ?true}, > > ?%% SSL options. > ?{cacertfile, ? ? "TEST_CA/ca-cert.pem"}, > ?{certfile, ? ? ? "TEST_CA/hosts/pacifico_server/cert.pem"}, > ?{keyfile, ? ? ? ?"TEST_CA/hosts/pacifico_server/key.pem"}, > ?{ssl_imp, ? ? ? ?new}, > ?{verify, ? ? ? ? verify_peer}, > ?{depth, ? ? ? ? ?1}, > ?{ciphers, ? ? ? ?ssl:cipher_suites()}, > ?{reuse_sessions, false} > ]. > > start() -> > ?io:format ("ssl:listen\n"), > ?{ok, Socket} = ssl:listen (10000, server_ssl_settings ()), > > ?io:format ("ssl:transport_accept\n"), > ?{ok, Client_Socket} = ssl:transport_accept (Socket), > > ?io:format ("ssl:ssl_accept\n"), > ?case ssl:ssl_accept (Client_Socket) of > ? ?ok -> > ? ? ?io:format ("ssl:ssl_accept: accepted\n"), > ? ? ?io:format ("ssl:close\n"), > ? ? ?ok = ssl:close (Client_Socket); > ? ?{error, Reason} -> > ? ? ?io:format ("ssl:ssl_accept: error ~w\n", [Reason]) > ?end, > > ?io:format ("ssl:close\n"), > ?ok = ssl:close (Socket). > > -- > > Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] > Eshell V5.7.4 ?(abort with ^G) > > application:start (sasl), > application:start (crypto), > application:start (ssl), > > -- > > 1> ssl:versions(). > [{ssl_app,"3.10.7"}, > ?{supported,[tlsv1,sslv3]}, > ?{available,[tlsv1,sslv3]}] > 2> sslserv_new:start(). > ssl:listen > ssl:transport_accept > ssl:ssl_accept > > Then, connecting using 'openssl s_client' using a known, verifiable test certificate: > > ** exception exit: {{function_clause,[{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{lists,foldl,3}]}, > ? ? ? ? ? ? ? ? ? ?{gen_fsm,sync_send_all_state_event, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? [<0.60.0>,started,infinity]}} > ? ? in function ?gen_fsm:sync_send_all_state_event/3 > ? ? in call from ssl:ssl_accept/2 > ? ? in call from sslserv_new:start/0 > 3> > =ERROR REPORT==== 3-May-2010::15:57:29 === > ** State machine <0.60.0> terminating > ** Last event in was {ssl_tls,undefined,22, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{3,1}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?<<1,0,0,133,3,1,0,93,0,0,0,32,0,0,57,0,0,56,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?53,0,0,136,0,0,135,0,0,132,0,0,22,0,0,19,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?10,7,0,192,0,0,51,0,0,50,0,0,47,0,0,69,0,0,68, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?0,0,65,3,0,128,0,0,5,0,0,4,1,0,128,0,0,21,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?18,0,0,9,6,0,64,0,0,20,0,0,17,0,0,8,0,0,6,4,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?128,0,0,3,2,0,128,198,211,40,132,76,120,105, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?20,171,188,10,216,42,133,0,122,173,212,152, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?167,161,137,84,183,0,215,233,12,153,221,99,235>>} (for all states) > ** When State == hello > ** ? ? ?Data ?== {state,server, > ? ? ? ? ? ? ? ? ? ? {#Ref<0.0.0.70>,<0.35.0>}, > ? ? ? ? ? ? ? ? ? ? gen_tcp,tcp,tcp_closed,"localhost",10000,#Port<0.1154>, > ? ? ? ? ? ? ? ? ? ? {ssl_options,[],verify_none,#Fun,false, > ? ? ? ? ? ? ? ? ? ? ? ? false,1,"TEST_CA/hosts/pacifico_server/cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? "TEST_CA/hosts/pacifico_server/key.pem",undefined, > ? ? ? ? ? ? ? ? ? ? ? ? undefined,"TEST_CA/ca-cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? [<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > ? ? ? ? ? ? ? ? ? ? ? ? #Fun,false,[]}, > ? ? ? ? ? ? ? ? ? ? {socket_options,list,0,0,0,false}, > ? ? ? ? ? ? ? ? ? ? {connection_states, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,0,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0,undefined,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<75,222,242,105,194,191,18,141,171,244,247, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 203,234,237,111,11,152,119,181,103,91,155, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 92,85,84,17,57,121,20,164,73,110>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,0,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0,undefined,0, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,0}, > ? ? ? ? ? ? ? ? ? ? ? ? {connection_state, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {security_parameters,undefined,0,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<75,222,242,105,194,191,18,141,171,244,247, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 203,234,237,111,11,152,119,181,103,91,155, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 92,85,84,17,57,121,20,164,73,110>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? undefined,undefined,undefined,undefined}}, > ? ? ? ? ? ? ? ? ? ? <<>>,<<>>, > ? ? ? ? ? ? ? ? ? ? {{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, > ? ? ? ? ? ? ? ? ? ? ? ? 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0>>, > ? ? ? ? ? ? ? ? ? ? ? <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, > ? ? ? ? ? ? ? ? ? ? ? ? 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116,97, > ? ? ? ? ? ? ? ? ? ? ? ? 120,95,116,111,111,108,115,45,49,46,54,46,52,47,101, > ? ? ? ? ? ? ? ? ? ? ? ? 98,105,110,47,115,115,108,95,104,97,110,100,115,104, > ? ? ? ? ? ? ? ? ? ? ? ? 97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}, > ? ? ? ? ? ? ? ? ? ? ?{<<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50, > ? ? ? ? ? ? ? ? ? ? ? ? 16,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,112,125,116,1,8,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 98,101,97,109,0,0,0,0,113,0,0,0,0,0,0,0,3,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,0,0,0>>, > ? ? ? ? ? ? ? ? ? ? ? <<1,35,69,103,137,171,205,239,254,220,186,152,118,84, > ? ? ? ? ? ? ? ? ? ? ? ? 50,16,240,225,210,195,0,0,0,0,0,0,0,0,121,110,116, > ? ? ? ? ? ? ? ? ? ? ? ? 97,120,95,116,111,111,108,115,45,49,46,54,46,52,47, > ? ? ? ? ? ? ? ? ? ? ? ? 101,98,105,110,47,115,115,108,95,104,97,110,100,115, > ? ? ? ? ? ? ? ? ? ? ? ? 104,97,107,101,46,98,101,97,109,0,0,0,193,1,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ? ? 0,0,185,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0>>}}, > ? ? ? ? ? ? ? ? ? ? [], > ? ? ? ? ? ? ? ? ? ? <<48,130,3,93,48,130,2,69,2,1,1,48,13,6,9,42,134,72,134, > ? ? ? ? ? ? ? ? ? ? ? 247,13,1,1,5,5,0,48,129,136,49,17,48,15,6,3,85,4,10,19, > ? ? ? ? ? ? ? ? ? ? ? 8,80,65,67,73,70,73,67,79,49,20,48,18,6,3,85,4,11,20,11, > ? ? ? ? ? ? ? ? ? ? ? 112,97,99,105,102,105,99,111,95,99,97,49,36,48,34,6,9, > ? ? ? ? ? ? ? ? ? ? ? 42,134,72,134,247,13,1,9,1,22,21,112,97,99,105,102,105, > ? ? ? ? ? ? ? ? ? ? ? 99,111,95,99,97,64,108,111,99,97,108,104,111,115,116,49, > ? ? ? ? ? ? ? ? ? ? ? 10,48,8,6,3,85,4,7,19,1,46,49,10,48,8,6,3,85,4,8,19,1, > ? ? ? ? ? ? ? ? ? ? ? 46,49,11,48,9,6,3,85,4,6,19,2,90,90,49,18,48,16,6,3,85, > ? ? ? ? ? ? ? ? ? ? ? 4,3,19,9,108,111,99,97,108,104,111,115,116,48,30,23,13, > ? ? ? ? ? ? ? ? ? ? ? 49,48,48,53,48,50,49,51,52,57,52,51,90,23,13,50,48,48, > ? ? ? ? ? ? ? ? ? ? ? 52,50,57,49,51,52,57,52,51,90,48,96,49,11,48,9,6,3,85,4, > ? ? ? ? ? ? ? ? ? ? ? 6,19,2,90,90,49,10,48,8,6,3,85,4,8,19,1,46,49,17,48,15, > ? ? ? ? ? ? ? ? ? ? ? 6,3,85,4,10,19,8,80,65,67,73,70,73,67,79,49,24,48,22,6, > ? ? ? ? ? ? ? ? ? ? ? 3,85,4,11,20,15,112,97,99,105,102,105,99,111,95,115,101, > ? ? ? ? ? ? ? ? ? ? ? 114,118,101,114,49,24,48,22,6,3,85,4,3,20,15,112,97,99, > ? ? ? ? ? ? ? ? ? ? ? 105,102,105,99,111,95,115,101,114,118,101,114,48,130,1, > ? ? ? ? ? ? ? ? ? ? ? 34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,130,1,15, > ? ? ? ? ? ? ? ? ? ? ? 0,48,130,1,10,2,130,1,1,0,240,234,138,84,118,47,107,232, > ? ? ? ? ? ? ? ? ? ? ? 150,182,200,67,1,35,15,48,208,88,231,11,213,21,249,110, > ? ? ? ? ? ? ? ? ? ? ? 226,145,33,249,255,250,114,58,255,247,130,249,140,60, > ? ? ? ? ? ? ? ? ? ? ? 108,201,2,98,26,254,16,213,173,238,140,201,39,75,7,97, > ? ? ? ? ? ? ? ? ? ? ? 209,120,94,80,69,11,72,117,162,83,84,194,57,232,106,19, > ? ? ? ? ? ? ? ? ? ? ? 2,196,52,255,102,220,178,30,82,85,10,96,118,20,104,238, > ? ? ? ? ? ? ? ? ? ? ? 54,214,183,157,110,205,247,220,236,34,209,225,95,113, > ? ? ? ? ? ? ? ? ? ? ? 244,195,193,56,177,196,204,248,203,210,172,236,124,75, > ? ? ? ? ? ? ? ? ? ? ? 60,246,172,183,76,9,253,155,20,101,63,66,12,109,213,186, > ? ? ? ? ? ? ? ? ? ? ? 167,95,2,197,100,120,22,94,247,229,2,22,95,54,216,42,78, > ? ? ? ? ? ? ? ? ? ? ? 230,204,144,123,218,153,113,128,155,236,88,228,171,169, > ? ? ? ? ? ? ? ? ? ? ? 59,165,2,122,196,149,208,179,249,5,86,38,39,217,79,193, > ? ? ? ? ? ? ? ? ? ? ? 7,121,41,193,201,3,43,109,225,62,195,67,173,248,3,245, > ? ? ? ? ? ? ? ? ? ? ? 81,210,197,193,236,3,91,150,57,142,97,34,18,138,104,212, > ? ? ? ? ? ? ? ? ? ? ? 252,188,6,165,221,221,243,166,115,40,247,179,191,163, > ? ? ? ? ? ? ? ? ? ? ? 127,190,211,153,161,229,113,220,178,216,48,240,116,133, > ? ? ? ? ? ? ? ? ? ? ? 103,20,107,225,88,214,163,63,233,4,175,50,61,6,22,240, > ? ? ? ? ? ? ? ? ? ? ? 78,107,145,2,3,1,0,1,48,13,6,9,42,134,72,134,247,13,1,1, > ? ? ? ? ? ? ? ? ? ? ? 5,5,0,3,130,1,1,0,103,56,251,221,227,238,147,176,66,136, > ? ? ? ? ? ? ? ? ? ? ? 67,183,114,184,232,52,77,105,35,127,218,87,140,246,244, > ? ? ? ? ? ? ? ? ? ? ? 131,178,205,126,183,13,38,12,90,3,172,190,31,142,5,170, > ? ? ? ? ? ? ? ? ? ? ? 202,43,229,222,118,97,167,201,150,182,54,41,67,173,234, > ? ? ? ? ? ? ? ? ? ? ? 202,139,219,152,255,34,15,191,247,240,37,69,210,46,137, > ? ? ? ? ? ? ? ? ? ? ? 148,86,105,182,112,77,238,106,8,115,135,239,117,148,12, > ? ? ? ? ? ? ? ? ? ? ? 71,65,61,149,149,238,4,8,48,118,236,135,158,183,156,215, > ? ? ? ? ? ? ? ? ? ? ? 132,122,46,139,35,81,172,60,217,218,157,198,183,10,142, > ? ? ? ? ? ? ? ? ? ? ? 41,67,186,92,144,238,232,144,223,150,33,7,141,63,164, > ? ? ? ? ? ? ? ? ? ? ? 230,173,145,48,36,152,147,32,83,156,37,23,191,250,58,51, > ? ? ? ? ? ? ? ? ? ? ? 26,228,110,156,248,226,26,247,42,10,180,228,98,202,249, > ? ? ? ? ? ? ? ? ? ? ? 31,223,122,26,83,94,47,134,135,202,150,140,201,178,178, > ? ? ? ? ? ? ? ? ? ? ? 137,77,116,240,36,174,17,44,221,75,61,53,196,213,107,79, > ? ? ? ? ? ? ? ? ? ? ? 230,160,158,202,223,113,180,19,11,255,19,247,161,211, > ? ? ? ? ? ? ? ? ? ? ? 221,185,204,232,155,16,207,175,83,7,237,175,220,121,90, > ? ? ? ? ? ? ? ? ? ? ? 170,96,198,200,43,146,250,75,86,97,65,60,250,211,255, > ? ? ? ? ? ? ? ? ? ? ? 104,133,172,217,210,30,144,86,10,90,96,157,85,79,59,249, > ? ? ? ? ? ? ? ? ? ? ? 206,138>>, > ? ? ? ? ? ? ? ? ? ? {session,undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? ? ? ? ? ? undefined,false,undefined}, > ? ? ? ? ? ? ? ? ? ? 20499,ssl_session_cache,undefined,undefined,false, > ? ? ? ? ? ? ? ? ? ? undefined,undefined, > ? ? ? ? ? ? ? ? ? ? {'RSAPrivateKey','two-prime', > ? ? ? ? ? ? ? ? ? ? ? ? 30412849349635586059878763176680338042806370928303169072580788733024438138591624114344661613699569307282187776490585714522517050565529617950655734681469324828571314711428867434319920530980711392758700590981596083356704738486321722811445451684075618202284684847604912244382636109829288466866694913244559971697849183994297490539908862341588923396172960090070282527346558060132865021448687169667310568206513992904712186265982889729037153800223846493957161193620543020688239671219777230989849538520312806141702778165525520364118176466217430310054602386812099554335308017341261109213665732076271625040081260558423830719377, > ? ? ? ? ? ? ? ? ? ? ? ? 65537, > ? ? ? ? ? ? ? ? ? ? ? ? 15199231804150546259658042484791966551963578848813856243560714914822915640833292103958537953118116384964424618816792863691594075974988046092098009948916868274565455708125634299346634071611627633962886925804052924579735878963676936497600941132620179791254257637877896314147191365262961154085247394502328039931767245176290407517156245147167225384726350170243685109653662879990481102526168621741076751484642682168826381020813101510533444174790316691177259394914972358200408962095948054841259799747816506752270496315220170738960924479086469481940145231690103702610522190880399648129761477972116477505211965989493531309937, > ? ? ? ? ? ? ? ? ? ? ? ? 176191409302436409962613455746475864610360333729834662591534394409663736050830385559280211933901474704302550960882948381498437634097559237504384892632519298195621737913663090630551501287500290638791916696994910572811737440764017948768162498168387518624603640493626338734757174408297008712871342547046062968773, > ? ? ? ? ? ? ? ? ? ? ? ? 172612555118571442807091000766318626155131399021717029980434951225715253087609998682422104752170804119306391847675331885937326037489735337859546704781797012286523769446860097366850215779124291431114338493212796775116102030799794358724497530713924646525414698841688697157985955712166061805209593917296689331549, > ? ? ? ? ? ? ? ? ? ? ? ? 115903444433627393782263321545723293331734054164222225210892027584256572449507905186866785127399184849045448785046396243696852820257472478863680691688255061778989645008679725395796822001414659353031068139231954233716215642251222085345576111525329549764925342157273276401618189772044808880926946913777711318945, > ? ? ? ? ? ? ? ? ? ? ? ? 160346862926570173155556405185673405256944925346630648110363303639494401757384328238794234360928308509443110543455974567280534799615104252084916968843795140271961900665652116021390071816425635325483939262810245627188737532006217565026586655932736202152482519271281991592202648637512700347912783277919685772993, > ? ? ? ? ? ? ? ? ? ? ? ? 153756145373284063246387226074130484888653092242571108769579800148375684177761146700330667363028796981671449245310468559005581499287242595226649317375919855635375141187450874362896818902873290546399632649342407712407283763334294537805123192425374231179698233819636663246910878178808591809105825780945571002589, > ? ? ? ? ? ? ? ? ? ? ? ? asn1_NOVALUE}, > ? ? ? ? ? ? ? ? ? ? undefined,undefined,#Ref<0.0.0.73>, > ? ? ? ? ? ? ? ? ? ? {<0.35.0>,#Ref<0.0.0.80>}, > ? ? ? ? ? ? ? ? ? ? 0,<<>>,true} > ** Reason for termination = > ** {function_clause,[{pubkey_cert,select_extension,[{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? {pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? {ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? {ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? {ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? {lists,foldl,3}]} > > =CRASH REPORT==== 3-May-2010::15:57:29 === > ?crasher: > ? ?initial call: ssl_connection:init/1 > ? ?pid: <0.60.0> > ? ?registered_name: [] > ? ?exception exit: {function_clause, > ? ? ? ? ? ? ? ? ? ? ? ?[{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ? ? {pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ? ? {ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ? ? {lists,foldl,3}]} > ? ? ?in function ?gen_fsm:terminate/7 > ? ?ancestors: [ssl_connection_sup,ssl_sup,<0.50.0>] > ? ?messages: [] > ? ?links: [<0.54.0>] > ? ?dictionary: [] > ? ?trap_exit: false > ? ?status: running > ? ?heap_size: 610 > ? ?stack_size: 24 > ? ?reductions: 1822 > ?neighbours: > > =SUPERVISOR REPORT==== 3-May-2010::15:57:29 === > ? ? Supervisor: {local,ssl_connection_sup} > ? ? Context: ? ?child_terminated > ? ? Reason: ? ? {function_clause, > ? ? ? ? ? ? ? ? ? ? [{pubkey_cert,select_extension, > ? ? ? ? ? ? ? ? ? ? ? ? ?[{2,5,29,35},asn1_NOVALUE]}, > ? ? ? ? ? ? ? ? ? ? ?{pubkey_cert,issuer_id,2}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_certificate,certificate_chain,4}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_handshake,certificate,3}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,certify_server,1}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,server_certify_and_key_exchange,1}, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,do_server_hello,2}, > ? ? ? ? ? ? ? ? ? ? ?{lists,foldl,3}]} > ? ? Offender: ? [{pid,<0.60.0>}, > ? ? ? ? ? ? ? ? ?{name,undefined}, > ? ? ? ? ? ? ? ? ?{mfa, > ? ? ? ? ? ? ? ? ? ? ?{ssl_connection,start_link, > ? ? ? ? ? ? ? ? ? ? ? ? ?[server,"localhost",10000,#Port<0.1154>, > ? ? ? ? ? ? ? ? ? ? ? ? ? {{ssl_options,[],verify_none,#Fun, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?false,false,1, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"TEST_CA/hosts/pacifico_server/cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"TEST_CA/hosts/pacifico_server/key.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?undefined,[],"TEST_CA/ca-cert.pem", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[<<0,10>>,<<0,47>>,<<0,5>>,<<0,4>>,<<0,9>>], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?#Fun,false,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{socket_options,list,0,0,0,false}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? <0.35.0>, > ? ? ? ? ? ? ? ? ? ? ? ? ? {gen_tcp,tcp,tcp_closed}]}}, > ? ? ? ? ? ? ? ? ?{restart_type,temporary}, > ? ? ? ? ? ? ? ? ?{shutdown,4000}, > ? ? ? ? ? ? ? ? ?{child_type,worker}] > > Any ideas what might be going on? > > Regards, > M > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From als@REDACTED Tue May 4 12:54:09 2010 From: als@REDACTED (Anthony Shipman) Date: Tue, 4 May 2010 21:54:09 +1100 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: <201005042054.10050.als@iinet.net.au> On Tue, 4 May 2010 05:57:19 am Sergey Samokhin wrote: > Hello! > > While reading turorials of how MongoDB documents are made in > Python/Ruby, I saw one thing I ever wanted to have in Erlang: > > {"author": "Mike", > "text": "My first blog post!", > "tags": ["mongodb", "python", "pymongo"], > "date": datetime.datetime.utcnow()} > > That is special syntax for dictionaries. This is not far off it dict:from_list([ {"author", "Mike"}, {"tags", ["mongodb", "python", "pymongo"]} ]) -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From g@REDACTED Tue May 4 14:27:14 2010 From: g@REDACTED (Garrett Smith) Date: Tue, 4 May 2010 07:27:14 -0500 Subject: [erlang-questions] Designing supervision trees In-Reply-To: <4BDFA13A.4090408@m5net.com> References: <4BDFA13A.4090408@m5net.com> Message-ID: Hi Bernard, On Mon, May 3, 2010 at 11:23 PM, Bernard Duggan wrote: > Hi list, > ? ?I'm in the process of going through the design of the supervision tree > for our application and it's rapidly becoming obvious to me that I could be > a lot clearer on how supervision trees are "meant" to be structured to make > a system as fault-tolerant as possible. ?Let me ask a couple of concrete > questions and maybe it will help: > > * Is it kosher to have a supervised process (say a gen_server) start its own > helper process(es) simply using spawn_link()? ?It seems like it should be > fine - any failure of either will propagate over the link, causing them both > to be shut down and the supervisor will then restart the main one which in > turn will restart the helper. ?I say it "seems like it should be fine", but > after reading all the supervisor and OTP docs I could lay my hands on I'm > not really sure if there isn't some good reason to avoid this arrangement. I think you'll find different opinions here. I prefer to use supervisors to start and manage processes. If a process needs to "spawn" something, it might make sense to call add_child on an appropriate supervisor. You can create a custom add_xxx_child function on your supervisor module to act somewhat as a factory function. This lets you fire-and-forget (in particular, using simple_one_for_one, which can remove children when they terminate) and avoids messing around with trap_exits in you gen_servers. > * Let's say I have two processes, A and B. ?The state of B is dependent on > the messages it has received from A. ?The particular example I'm dealing > with is a process (B) who is responsible for starting and stopping apps and > another (A) which is responsible for synchronising data with a remote store. > ?The apps should be started when we are synced and stopped if we lose > connection with the store. ?I don't necessarily want to merge them in to one > process because A needs to be relatively responsive to the remote store, but > the process of starting/stopping apps can take some time. ?There may be a > much better way to arrange this, but I'm not exactly sure what it is... > So we're up and running, remote store is synced, all the apps are running, > and B crashes. ?I'm trying to figure out the "right" way to manage recovery > - possibilities I can think of: > - Have A and B under a one-for-all supervisor so that we just nuke the > broader state and start it all again (seems like we should be able to > recover with less impact than this). This would be my starting point. If you have efficiency concerns about tearing everything down and rebuilding it, consider breaking what's expensive *and* independent into a separate supervisory hierarchy -- or its own app. > - Have B's state stored in an ETS table owned by its parent so that it can > recover into it's previous state (that seems far too much like global data > to me). If it's costly to grab the external state and you need the process state to recover, this would make sense, I think. Though dets or mnesia might be a better option for recovery. This might be that independent component that you move outside your A/B hierarchy that would survive crashes in those processes. > - Have B query A for the current state on startup - that would work, except > that it leaves us with multiple communication methods between A and B, one > where B asks for the state and one where A pushes state updates - that seems > a little redundant (and like extra code to maintain which nobody wants). > - Get rid of B entirely and perhaps have A spawn a temporary processes to do > app start/stop when required - seems a little messy, and prone to race > conditions if we do multiple start/stop operations in quick succession... Yes, that's why the first option (total restart) makes more sense, IMO. You don't need to get into the game of trying to restore your state piecemeal. You just trash it all and start from scratch. > Any thoughts (and pointers to docs that deal with this stuff) are > appreciated. > > As always, thanks if you read this far :) > > Cheers, > > Bernard Garrett From mevans@REDACTED Tue May 4 16:25:25 2010 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 4 May 2010 10:25:25 -0400 Subject: FW: pg2 is broken R13B04 Message-ID: Hi, I sent this to erlang-bugs too, but for some reason it didn't get in the list? .... So after more tests I have seen that pg2 is definitely not working as intended. It appears that the root problem is how a new instantiation of pg2 within a cluster of Erlang nodes gets its data. The following sequences of events occur. 1) All nodes do a net_kernel:monitor_nodes(true) in the init function of pg2. 2) The new instance of pg2 will send {new_pg2, node()} to all other nodes in the pool. 3) The new instance of pg2 will send {nodeup, Node} to itself (where nodes is a list of nodes()). What it appears is that when only 2 nodes are in the pool things are generally ok. However, the synchronization process gets muddied when there are many members. The process of updating the nodes is that upon receipt of {new_pg2,Node} or {nodeup,Node} to literally go through the table of pids in the ets pg2_table and build a list similar to: [proxy_micro_cache,[<6325.319.0>,<6324.324.0>]]] This is dispatched to the new pg2 instance. The problem is every node does that, so the new pg2 instance will end up with a table like: [proxy_micro_cache, [<6437.319.0>,<6437.319.0>,<6437.319.0>,<6437.319.0>, <6437.319.0>,<6437.319.0>,<6436.324.0>,<6436.324.0>, <6436.324.0>,<6436.324.0>,<6436.324.0>,<6436.324.0>]]] Where there are many instances for each Pid since each node has sent its copy of the data, causing that process to be replicated many times (i.e. the call to ets:update_counter in pg2:join_group)!!! This problem is compounded further on nodes that join later, or when a VM stops and is restarted. An additional problem arises when another process in the new group sends pg2:join on its own. In that case there is a timing window whereby the new instance could get that new entry more than once. My recommendation is: 1) Have a new pg2:join function called pg2:join_once. In this case a process will never be permitted to have more than 1 join. 2) When a new node joins one could either select only one node to get its data from, or have all nodes in the system send the result of ets:tab2list(pg2_table) to the new node, then have that data inserted directly into its local ets table, as opposed to going through the process of join_group (possibly with the additional step erlang:monitor/2). In this way a process that has been registered more than once would be inserted into the local ets table as a single operation as opposed to many times. 3) Possibly defer new requests to pg2:join on the new instance until synchronization is complete. I understand that gproc is on the way, but I suspect that pg2 does need fixing. Regards Matt From dougedmunds@REDACTED Tue May 4 16:42:59 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Tue, 04 May 2010 07:42:59 -0700 Subject: [erlang-questions] Process unexpectedly locks up In-Reply-To: <4BDFB397.4040206@m5net.com> References: <4BDFA80C.8010809@gmail.com> <4BDFB397.4040206@m5net.com> Message-ID: <4BE03273.1010000@gmail.com> Hello Sam, Your suggestion for using a {down, X} handler works. Thank you! Modify the code this way: 1. Remove the original 'down' handler from the me_loop. 2. Replace with these two handlers: -------------- down -> case length(K) of 0 -> io:format("--No kids~n"); 1 -> [Head |_] = K, Head ! {self(), info_request}; _ -> Out = indexlist(1, K, []), ok = io:format("Use me ! {down, Integer} to select: ~p~n", [Out]) end, NM = M, NK = K, NP = P; {down, Input} -> % use after down displays the choices case is_integer(Input) of true -> Out = indexlist(1, K, []), Pick = pick_pid(Out,Input), case is_pid(Pick) of true -> Pick ! {self(), info_request}; _ -> io:format("That number is not on the list~n") end; _ -> io:format("Must enter an integer~n") end, NM = M, NK = K, NP = P; ------------ --- Doug Edmunds On 5/3/2010 10:41 PM, Sam Bobroff wrote: > * add a handler for a {down, X} message that takes choice X as if it had > been returned from get_user_input() in the old code. From comptekki@REDACTED Tue May 4 18:54:22 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 4 May 2010 10:54:22 -0600 Subject: strip end off of string Message-ID: I see that fist ++ rest below will match the first part of the string and the put the rest in "Rest": -module(t). -export([gett/0]). gett() -> t(""). t(" t2(Rest). t2(First ++ "\">") -> io:format("~s~n", [First]). But in t2 is there something like that? I can do string:substr(Str, 1, length(Str)-2). I'm guessing from what I've seen in examples, that erlang is a first | rest way of doing things. or use some re:run expressoin. I'm trying to pull out some_string. I tried some experiments with some_string\">: G="some_string\">". "some_string\">" 110> re:run("(?.*)\">", [caseless,{capture,['VAL'],list}]). ** exception error: bad argument in function re:run/2 called as re:run("(?.*)\">",[caseless,{capture,['VAL'],list}]) 112> erlang:erl_scan(G). ** exception error: undefined function erlang:erl_scan/1 113> erl_scan:string(G). {error,{1,erl_scan,{string,34,">"}},1} thx, -wes From zuav@REDACTED Tue May 4 19:18:39 2010 From: zuav@REDACTED (Alexander Zhukov) Date: Tue, 04 May 2010 21:18:39 +0400 Subject: Why there is no way to open file with O_EXCL mode? Message-ID: <87y6fzbn68.fsf@yandex.ru> Hi! Man page for file module do not lists 'excl' as a possible mode to open/2 function. As I understood it's an old problem. But what I do not understood completely is that why there is no 'excl' mode? Really I'm very interested in getting an answer to the question. -- Alexander Zhukov From tuncer.ayaz@REDACTED Tue May 4 20:15:18 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 4 May 2010 20:15:18 +0200 Subject: [erlang-questions] Why there is no way to open file with O_EXCL mode? In-Reply-To: <87y6fzbn68.fsf@yandex.ru> References: <87y6fzbn68.fsf@yandex.ru> Message-ID: On Tue, May 4, 2010 at 7:18 PM, Alexander Zhukov wrote: > Hi! > > Man page for file module do not lists 'excl' as a possible mode to > open/2 function. > > As I understood it's an old problem. But what I do not understood > completely is that > > ? ? ? ? ? why there is no 'excl' mode? > > Really I'm very interested in getting an answer to the question. Michael Santos wrote a patch for this and it's currently included in the 'pu' branch. It may end up in R14. From alisdairsullivan@REDACTED Tue May 4 21:05:29 2010 From: alisdairsullivan@REDACTED (Alisdair Sullivan) Date: Tue, 4 May 2010 12:05:29 -0700 Subject: Binary Matching Restrictions Message-ID: <796AEBFE-728D-45BE-88CD-CF3CE7AE903B@yahoo.ca> Curious as to why functions had to end with a tail call to a function that performs a binary match instruction in order for optimizations to be applied, I was reading beam_bsm.erl in compiler-4.6.5. The comments for btb_reaches_match/3 seem to imply that this is not actually necessary. Only that the match context is not copied or passed to a BIF. I tried to trace the code to get an idea for why the optimization is not applied (despite the comments indicating it should be), but it is still unclear to me. Any documentation I should read to better understand what's going on here? Thankyou, Alisdair. From colm.dougan@REDACTED Tue May 4 22:04:48 2010 From: colm.dougan@REDACTED (Colm Dougan) Date: Tue, 4 May 2010 13:04:48 -0700 Subject: [erlang-questions] strip end off of string In-Reply-To: References: Message-ID: On Tue, May 4, 2010 at 9:54 AM, Wes James wrote: > I see that fist ++ rest below will match the first part of the string > and the put the rest in "Rest": > > -module(t). > > -export([gett/0]). > > gett() -> t(""). > > t(" > ? ? ? ?t2(Rest). > > t2(First ++ "\">") -> > ? ? ?io:format("~s~n", [First]). > > > But in t2 is there something like that? Bearing in mind that: extract_foo("Foo:" ++ Rest) is just syntax sugar for : extract_foo([$F, $o, $o, $: | Rest]) You could write: t(" t2(lists:reverse(Rest)). t2([$>, $" | Prefix]) -> lists:reverse(Prefix). (not necessarily recommended but I'm mentioning it for the purposes of illustration). > I tried some experiments with some_string\">: > > G="some_string\">". > "some_string\">" > 110> re:run("(?.*)\">", [caseless,{capture,['VAL'],list}]). > ** exception error: bad argument > ? ? in function ?re:run/2 > ? ? ? ?called as re:run("(?.*)\">",[caseless,{capture,['VAL'],list}]) How about: Eshell V5.7.5 (abort with ^G) 1> Str = "". "" 2> re:run(Str, "VALUE=\"(\\w+)", [{capture,[1],list}]). {match,["some_string"]} Colm From mail@REDACTED Wed May 5 00:02:57 2010 From: mail@REDACTED (Tim Fletcher) Date: Tue, 4 May 2010 15:02:57 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: References: Message-ID: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> > My proposal for 'frames' has been sitting around for years. > I would write that as > ? ? ? ? <{ author ~ "Mike" > ? ? ? ? ?, text ? ~ "My first blog post!" > ? ? ? ? ?, tags ? ~ ["mongodb","python","pymongo"] > ? ? ? ? ?, date ? ~ erlang:now() > ? ? ? ? }> As much as I dislike the double bracketing I can see the necessity. Is there any reason why using the more common choice of ":" to delimit the keys and values would be a problem (instead of "~")? For example: <{author: "Mike", text: "My first blog post!"...}> Tim From tjgillies@REDACTED Wed May 5 04:54:15 2010 From: tjgillies@REDACTED (tyler gillies) Date: Tue, 4 May 2010 19:54:15 -0700 Subject: logging out of shell without shutting down VM Message-ID: I googled for awhile on this but all the examples i found to exit out of shell also shut down the VM -- Everyone Loves Tea http://www.everyonelovestea.com From g9414002.pccu.edu.tw@REDACTED Wed May 5 05:03:17 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Wed, 5 May 2010 11:03:17 +0800 Subject: [SWIPL] 8 Queens problem for a 3D chessboard In-Reply-To: <28451926.post@talk.nabble.com> References: <28451926.post@talk.nabble.com> Message-ID: Step 0. Correct the original program. Step 1. Use board/1 and queens/1 to produce 3 candidate answer sets, each one for one dimension. Step 2. Take every one answer from 3 candidate answer sets, and check if all chesses existing in the first answer set match corresponding ones in the second answer set, while ones in the later answer set match corresponding ones in the third answer set. This is a quick answer. Sorry for sort of inaccurate. On Wed, May 5, 2010 at 3:17 AM, xlacrimx wrote: > > I have the code for an 8 by 8 chessboard for the 8 Queens problem. How > could > I modify it to make it 8 by 8 by 8? Here's the code: > > queens([]). > queens(Row/Col|Rest]):-queens(Rest), member(Col,[1,2,3,4,5,6,7,8]), > safe(Row/Col,Rest). > > > safe(Anything,[]). > safe(Row/Col,[Row1/Col1|Rest]):-Col =\= Col1, Col1 - Col =\= Row1 - Row, > Col1 - Col =\= Row - Row1, safe(Row/Col, > Rest). > > member(X,[X|Tail]). > member(X,[Head|Tail]):-member(X|Tail). > > board([1/C1, 2/C2, 3/C3, 4/C4, 5/C5, 6/C6, 7/C7, 8/C8]). > -- > View this message in context: > http://old.nabble.com/8-Queens-problem-for-a-3D-chessboard-tp28451926p28451926.html > Sent from the SWI Prolog mailing list archive at Nabble.com. > > _______________________________________________ > SWI-Prolog mailing list > SWI-Prolog@REDACTED > https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog > From g9414002.pccu.edu.tw@REDACTED Wed May 5 05:04:18 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Wed, 5 May 2010 11:04:18 +0800 Subject: [SWIPL] 8 Queens problem for a 3D chessboard In-Reply-To: References: <28451926.post@talk.nabble.com> Message-ID: Sorry, Erlang guys. I post this article in a wrong mailing list. I'm sorry. 2010/5/5 ??? (Yau-Hsien Huang) > Step 0. Correct the original program. > > Step 1. Use board/1 and queens/1 to produce 3 candidate answer > sets, each one for one dimension. > > Step 2. Take every one answer from 3 candidate answer sets, and > check if all chesses existing in the first answer set match > corresponding ones in the second answer set, while ones in the > later answer set match corresponding ones in the third answer set. > > This is a quick answer. Sorry for sort of inaccurate. > > On Wed, May 5, 2010 at 3:17 AM, xlacrimx wrote: > >> >> I have the code for an 8 by 8 chessboard for the 8 Queens problem. How >> could >> I modify it to make it 8 by 8 by 8? Here's the code: >> >> queens([]). >> queens(Row/Col|Rest]):-queens(Rest), member(Col,[1,2,3,4,5,6,7,8]), >> safe(Row/Col,Rest). >> >> >> safe(Anything,[]). >> safe(Row/Col,[Row1/Col1|Rest]):-Col =\= Col1, Col1 - Col =\= Row1 - Row, >> Col1 - Col =\= Row - Row1, safe(Row/Col, >> Rest). >> >> member(X,[X|Tail]). >> member(X,[Head|Tail]):-member(X|Tail). >> >> board([1/C1, 2/C2, 3/C3, 4/C4, 5/C5, 6/C6, 7/C7, 8/C8]). >> -- >> View this message in context: >> http://old.nabble.com/8-Queens-problem-for-a-3D-chessboard-tp28451926p28451926.html >> Sent from the SWI Prolog mailing list archive at Nabble.com. >> >> _______________________________________________ >> SWI-Prolog mailing list >> SWI-Prolog@REDACTED >> https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog >> > > From bengt.kleberg@REDACTED Wed May 5 06:54:54 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 05 May 2010 06:54:54 +0200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> Message-ID: <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Would not using ":" make it look like a function call (to another module)? bengt On Wed, 2010-05-05 at 00:02 +0200, Tim Fletcher wrote: > > My proposal for 'frames' has been sitting around for years. > > I would write that as > > <{ author ~ "Mike" > > , text ~ "My first blog post!" > > , tags ~ ["mongodb","python","pymongo"] > > , date ~ erlang:now() > > }> > > As much as I dislike the double bracketing I can see the necessity. Is > there any reason why using the more common choice of ":" to delimit > the keys and values would be a problem (instead of "~")? > > For example: <{author: "Mike", text: "My first blog post!"...}> > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bgustavsson@REDACTED Wed May 5 07:53:20 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 5 May 2010 07:53:20 +0200 Subject: [erlang-questions] Binary Matching Restrictions In-Reply-To: <796AEBFE-728D-45BE-88CD-CF3CE7AE903B@yahoo.ca> References: <796AEBFE-728D-45BE-88CD-CF3CE7AE903B@yahoo.ca> Message-ID: On Tue, May 4, 2010 at 9:05 PM, Alisdair Sullivan wrote: > Curious as to why functions had to end with a tail call to a function that performs a binary match instruction in order for optimizations to be applied, On what do you base that claim? > Any documentation I should read to better understand what's going on here? Yes, the Efficiency Guide: http://www.erlang.org/doc/efficiency_guide/binaryhandling.html Section 4.3 is about matching binaries, and it also describes the bin_opt_info compiler option that will instruct the compiler to print information about binary matching optimizations or lack thereof. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Wed May 5 09:17:36 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 5 May 2010 09:17:36 +0200 Subject: [erlang-questions] logging out of shell without shutting down VM In-Reply-To: References: Message-ID: <20100505071736.GA14607@erix.ericsson.se> On Tue, May 04, 2010 at 07:54:15PM -0700, tyler gillies wrote: > I googled for awhile on this but all the examples i found to exit out > of shell also shut down the VM For the full (?) explanation of the right (?) way: http://erlang.org/doc/system_principles/ (esp. Starting a Target System) Might do what you want but be to complicated. For a completely different approach: http://erlang.org/doc/man/erl.html see the -detached flag Does not do what you say you want but might solve your actual problem depending on what it is. > > -- > Everyone Loves Tea > http://www.everyonelovestea.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From huss01@REDACTED Wed May 5 09:17:58 2010 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Wed, 5 May 2010 09:17:58 +0200 Subject: [erlang-questions] logging out of shell without shutting down VM In-Reply-To: References: Message-ID: What you can do is to start an Erlang node which provides a remote shell to an existing Erlang node. This shell can then be stopped without affecting the original node. The easiest way of doing this is by using the -remsh option of erl. Example: Start the erlang node which you want to have running: > erl -sname server -setcookie testcookie -noshell ... (server@REDACTED)1> Elsewhere > erl -sname shell -remsh server@REDACTED -setcookie testcookie ... (server@REDACTED)1> ^G User switch command --> q And the server node should still be running. For more advanced usage, you might want to look into run_erl and to_erl. Regards, /H?kan Huss On Wed, May 5, 2010 at 04:54, tyler gillies wrote: > I googled for awhile on this but all the examples i found to exit out > of shell also shut down the VM > > -- > Everyone Loves Tea > http://www.everyonelovestea.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mail@REDACTED Wed May 5 09:32:38 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 5 May 2010 00:32:38 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: > Would not using ":" make it look like a function call > (to another module)? Yes, if the values are atoms and you don't put spaces before/after the semicolon. Whilst that would be similar to a mod:fun call, there are no brackets, so technically there's no ambiguity. Currently that usage of the semicolon is illegal syntax, so presumably it could be made to work. And to make it really look like a function call you'd have to be using keys and values that are similar to those used to identify modules and functions, like this: <{erlang:list_to_integer}>. I think both of these look sufficiently different from a mod:fun call though: <{erlang: list_to_integer}>. <{erlang : list_to_integer}>. Maybe it's my exposure to languages that use the semicolon for this purpose, but it feels more natural than "~". Tim From tjgillies@REDACTED Wed May 5 09:37:50 2010 From: tjgillies@REDACTED (tyler gillies) Date: Wed, 5 May 2010 00:37:50 -0700 Subject: [erlang-questions] logging out of shell without shutting down VM In-Reply-To: References: Message-ID: using run_erl and to_erl, thanks On Wed, May 5, 2010 at 12:17 AM, H?kan Huss wrote: > What you can do is to start an Erlang node which provides a remote > shell to an existing Erlang node. This shell can then be stopped > without affecting the original node. The easiest way of doing this is > by using the -remsh option of erl. > > Example: > > Start the erlang node which you want to have running: > >> erl -sname server -setcookie testcookie -noshell > ... > (server@REDACTED)1> > > > Elsewhere > >> erl -sname shell -remsh server@REDACTED -setcookie testcookie > ... > (server@REDACTED)1> ^G > User switch command > ?--> q > > And the server node should still be running. > > For more advanced usage, you might want to look into run_erl and to_erl. > > Regards, > /H?kan Huss > > On Wed, May 5, 2010 at 04:54, tyler gillies wrote: >> I googled for awhile on this but all the examples i found to exit out >> of shell also shut down the VM >> >> -- >> Everyone Loves Tea >> http://www.everyonelovestea.com >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > -- Everyone Loves Tea http://www.everyonelovestea.com From lovei.laszlo@REDACTED Wed May 5 10:16:09 2010 From: lovei.laszlo@REDACTED (Laszlo Lovei) Date: Wed, 5 May 2010 01:16:09 -0700 (PDT) Subject: closure on function argument In-Reply-To: <4BDC7512.60604@tmit.bme.hu> References: <4BDC7512.60604@tmit.bme.hu> Message-ID: On May 1, 8:38?pm, Zoltan Lajos Kis wrote: > > If I have /F = fun(Y) -> fun(Y) -> ok end end/, and /G = F('z')/, I > would expect /G/ to be /fun('z') -> ok end/. [...] > A workaround I found is using guards: /F = fun(Y) -> fun(X) when X == Y > -> ok end end/. Just a side note: you must use =:= to get the exact semantics of fun('z') -> ok end. Laszlo From zuav@REDACTED Wed May 5 10:01:33 2010 From: zuav@REDACTED (Alexander Zhukov) Date: Wed, 05 May 2010 12:01:33 +0400 Subject: [erlang-questions] Why there is no way to open file with O_EXCL mode? In-Reply-To: (Tuncer Ayaz's message of "Tue, 4 May 2010 20:15:18 +0200") References: <87y6fzbn68.fsf@yandex.ru> Message-ID: <87sk66bwv6.fsf@yandex.ru> >>>>> Tuncer Ayaz (TA) writes: TA> On Tue, May 4, 2010 at 7:18 PM, Alexander Zhukov wrote: >> Man page for file module do not lists 'excl' as a possible mode to >> open/2 function. >> >> As I understood it's an old problem. But what I do not understood >> completely is that >> >> ? ? ? ? ? why there is no 'excl' mode? >> >> Really I'm very interested in getting an answer to the question. TA> Michael Santos wrote a patch for this and it's currently included TA> in the 'pu' branch. It may end up in R14. Great! Thank you for your answer. -- Alexander Zhukov From mayamatakeshi@REDACTED Wed May 5 10:31:04 2010 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Wed, 5 May 2010 17:31:04 +0900 Subject: gen_udp:send and concurrent access to the same Socket Message-ID: Hello, I'm testing with gen_udp. I want to pass the same Socket to several different processes so that they can use it to send messages to an external endpoint from time to time. The processes don't care about responses for the messages they send. So, is it OK to call gen_udp:send/4 from more than one process passing the same Socket? My doubt is if gen_udp:send/4 takes care of concurrent access or if I should do it myself, sending the packet to be transmitted as a message to a process that owns the Socket. br, takeshi From bengt.kleberg@REDACTED Wed May 5 10:47:32 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 05 May 2010 10:47:32 +0200 Subject: [erlang-questions] gen_udp:send and concurrent access to the same Socket In-Reply-To: References: Message-ID: <1273049252.5047.14.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, Any processes can send on a socket. Only the owner can receive on a socket. >From a design point of view it would seem better to me with a single process that does all the sending. If that is not possible given your performance requirements (measure) you can send from all processes. bengt On Wed, 2010-05-05 at 10:31 +0200, mayamatakeshi wrote: > Hello, > I'm testing with gen_udp. > I want to pass the same Socket to several different processes so that they > can use it to send messages to an external endpoint from time to time. The > processes don't care about responses for the messages they send. > So, is it OK to call gen_udp:send/4 from more than one process passing the > same Socket? > My doubt is if gen_udp:send/4 takes care of concurrent access or if I should > do it myself, sending the packet to be transmitted as a message to a process > that owns the Socket. > > br, > takeshi From rvirding@REDACTED Wed May 5 12:20:14 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 5 May 2010 12:20:14 +0200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: The problem is that syntax has to be interpreted by two different parsers: programmers who read the code and the language parser. While programmers can often infer what you mean the parser can only follow strict rules. This means that there can be no ambiguities. While your examples might work if the value is a simple constant or variable allowing any legal general expression will cause problems. How would you interpret <{erlang:list_to_integer("123")}> ? Note that white space is usually ignored so making white space significant here would be creating a special case for just this. There is no difference between erlang:list_to_integer and erlang : list_to_integer. Even if you grab the first atom + colon as key name you can still get some funny combinations, for example <{erlang:erlang:list_to_integer("123")}>. The parser would have no problems with this, it would blindly follow the rules, but a programmer might start to wonder. This was the main reason that = is used in the record syntax. While ~ may look strange using : will cause problems. How about =? Though this may force you to parenthesize some expressions. Robert On 5 May 2010 09:32, Tim Fletcher wrote: >> Would not using ":" make it look like a function call >> (to another module)? > > Yes, if the values are atoms and you don't put spaces before/after the > semicolon. Whilst that would be similar to a mod:fun call, there are > no brackets, so technically there's no ambiguity. Currently that usage > of the semicolon is illegal syntax, so presumably it could be made to > work. And to make it really look like a function call you'd have to be > using keys and values that are similar to those used to identify > modules and functions, like this: > > ?<{erlang:list_to_integer}>. > > I think both of these look sufficiently different from a mod:fun call > though: > > ?<{erlang: list_to_integer}>. > > ?<{erlang : list_to_integer}>. > > Maybe it's my exposure to languages that use the semicolon for this > purpose, but it feels more natural than "~". > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From als@REDACTED Wed May 5 12:20:34 2010 From: als@REDACTED (Anthony Shipman) Date: Wed, 5 May 2010 21:20:34 +1100 Subject: The ~g format Message-ID: <201005052020.34622.als@iinet.net.au> Why do these differ? 34> lists:flatten(io_lib:format("~.2g", [23.223466])). "2.3e+1" 35> lists:flatten(io_lib:format("~.2f", [23.223466])). "23.22" 36> lists:flatten(io_lib:format("~.3g", [23.223466])). "23.2" The manual says: "If the absolute value of the float does not allow it to be written in the f format with the desired number of significant digits, it is also written in the e format." and that is probably what is happening but I can't match that to the behaviour of the code. In the first case is it trying to avoid writing "23." or just "23"? This is in R12B04 -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From rvirding@REDACTED Wed May 5 12:30:24 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 5 May 2010 12:30:24 +0200 Subject: [erlang-questions] The ~g format In-Reply-To: <201005052020.34622.als@iinet.net.au> References: <201005052020.34622.als@iinet.net.au> Message-ID: The ~g, ~e and ~f formats all print the float in legal Erlang float syntax which requires the decimal point and digits on both sides of it. So 23 , 23. , and 23e+1 are not legal floats. Robert On 5 May 2010 12:20, Anthony Shipman wrote: > Why do these differ? > > 34> lists:flatten(io_lib:format("~.2g", [23.223466])). > "2.3e+1" > 35> lists:flatten(io_lib:format("~.2f", [23.223466])). > "23.22" > 36> lists:flatten(io_lib:format("~.3g", [23.223466])). > "23.2" > > The manual says: > ? ?"If the absolute value of the float does not allow it to be written in the > f format with the desired number of significant digits, it is also written in > the e format." > and that is probably what is happening but I can't match that to the behaviour > of the code. ?In the first case is it trying to avoid writing "23." or > just "23"? > > This is in R12B04 > -- > Anthony Shipman ? ? ? ? ? ? ? ? ? ?Mamas don't let your babies > als@REDACTED ? ? ? ? ? ? ? ? ? grow up to be outsourced. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rvirding@REDACTED Wed May 5 12:36:45 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 5 May 2010 12:36:45 +0200 Subject: [erlang-questions] closure on function argument In-Reply-To: References: <4BDC7512.60604@tmit.bme.hu> Message-ID: Yes, I know. The reason is rather simple in that there were different influences at the times when these things were implemented. So originally we were influenced by Prolog which resulted in unscoped variables and at that time Erlang had very few "functional" constructs. Later when we started adding more functional constructs, funs, comprehensions and higher-order functions, we were influenced by more "standard" functional languages and used their scoping rules. Regrettable I admit, Robert On 3 May 2010 01:33, Richard O'Keefe wrote: > > On May 2, 2010, at 8:56 AM, Robert Virding wrote: > >> The variables in the arguments of a fun, Y in your example, are always >> "fresh" variables and never imported from the funs environment. > > For me personally, this is perhaps THE most confusing thing > about Erlang. > > I have *no* problem with the same rule in ML or Haskell because they > apply it *consistently*: every pattern everywhere introduces fresh > variables, it is never ever possible in those languages for a > variable occurrence in a pattern to be an applied occurrence; all > pattern variables are binding occurrences. ?This applies to OCaml and > Clean as well. > > But Erlang does NOT apply this rule consistently. > Variables in a pattern of a 'case' are *not* fresh, they *may* be > applied occurrences. ?And even in a fun, *some* variable occurrences > may be applied ones: ?fun (Y,Y) -> ... end. > > It's especially confusing because variables in fun bodies and guards > are imported as usual, so they are clearly in scope. > > It would be *so* nice if ALL variables in Erlang followed exactly > the same rules. > > Here's a little example. ?Suppose someone has > ? ? ? ?case E of P -> B end > but doesn't want any variable bindings in B to be exported. > The obvious thing to do is to rewrite it as > ? ? ? ?(fun (P) -> B end)(E) > but because Erlang is inconsistent here, that does not work. > (You can patch around it with fun, case, and a new variable, > but that has problems of its own.) > > I have never been able to see any merit in this rule; its effect > on me is to make it harder to write funs because I always have to > watch out for cases where the expected thing won't happen and it > has to be patched around. > > > > From alessandro.sivieri@REDACTED Wed May 5 13:01:43 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 5 May 2010 13:01:43 +0200 Subject: [erlang-questions] Designing supervision trees In-Reply-To: References: <4BDFA13A.4090408@m5net.com> Message-ID: 2010/5/4 Garrett Smith > I think you'll find different opinions here. I prefer to use > supervisors to start and manage processes. If a process needs to > "spawn" something, it might make sense to call add_child on an > appropriate supervisor. You can create a custom add_xxx_child function > on your supervisor module to act somewhat as a factory function. > > This lets you fire-and-forget (in particular, using > simple_one_for_one, which can remove children when they terminate) and > avoids messing around with trap_exits in you gen_servers. > > This is very interesting, essentially because I am doing the same thing as Bernard; but, for what I have understood from the supervisor documentation, if I want a child to be supervised, it has to implement one of the four behaviours, doesn't it? So, if my child doesn't do that (it is simply a process which executes a single function in loop), how can I attach it to a supervisor? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From mail@REDACTED Wed May 5 14:17:17 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 5 May 2010 05:17:17 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: > While your examples might work if the value is a simple constant or > variable allowing any legal general expression will cause problems. > How would you interpret <{erlang:list_to_integer("123")}> ? Note that > white space is usually ignored so making white space significant here > would be creating a special case for just this. You're right, that's definitely not workable. > This was the main reason that = is used in the record syntax. While ~ > may look strange using : will cause problems. How about =? Though this > may force you to parenthesize some expressions. I think = is fine aesthetically, but Richard notes in his frames proposal that it could also be confused with existing syntax that uses =. Adopting the Ruby style "arrows" would be another option: <{author=>"Mike", erlang=>list_to_integer("123")}> Looks clear enough, but might be a bit too verbose for typing out? Tim From hynek@REDACTED Wed May 5 15:08:13 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 5 May 2010 15:08:13 +0200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: On Wed, May 5, 2010 at 2:17 PM, Tim Fletcher wrote: >> While your examples might work if the value is a simple constant or >> variable allowing any legal general expression will cause problems. >> How would you interpret <{erlang:list_to_integer("123")}> ? Note that >> white space is usually ignored so making white space significant here >> would be creating a special case for just this. > > You're right, that's definitely not workable. > >> This was the main reason that = is used in the record syntax. While ~ >> may look strange using : will cause problems. How about =? Though this >> may force you to parenthesize some expressions. > > I think = is fine aesthetically, but Richard notes in his frames > proposal that it could also be confused with existing syntax that uses > =. Adopting the Ruby style "arrows" would be another option: > > ?<{author=>"Mike", erlang=>list_to_integer("123")}> > > Looks clear enough, but might be a bit too verbose for typing out? Just to be correct it is not Ruby style but Perl style and it was found brief enough even in Perl sense. > > Tim > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From mail@REDACTED Wed May 5 17:39:14 2010 From: mail@REDACTED (Tim Fletcher) Date: Wed, 5 May 2010 08:39:14 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <9821c743-ddef-41d5-9c9b-ed97792bafce@r11g2000yqa.googlegroups.com> > Just to be correct it is not Ruby style but Perl style Surely it's still in the style of Ruby, even if the style originated elsewhere? :) > it was found brief enough even in Perl sense. Sure, I like it, but the { : } combo is half as many characters for the delimiters as the <{ => }> combo. Just trying to compare the alternatives. Syntax aside, frames/structs would be nice to have. Tim From g@REDACTED Wed May 5 20:45:20 2010 From: g@REDACTED (Garrett Smith) Date: Wed, 5 May 2010 13:45:20 -0500 Subject: [erlang-questions] Designing supervision trees In-Reply-To: References: <4BDFA13A.4090408@m5net.com> Message-ID: Bernard, On Wed, May 5, 2010 at 6:01 AM, Alessandro Sivieri wrote: > 2010/5/4 Garrett Smith >> >> I think you'll find different opinions here. I prefer to use >> supervisors to start and manage processes. If a process needs to >> "spawn" something, it might make sense to call add_child on an >> appropriate supervisor. You can create a custom add_xxx_child function >> on your supervisor module to act somewhat as a factory function. >> >> This lets you fire-and-forget (in particular, using >> simple_one_for_one, which can remove children when they terminate) and >> avoids messing around with trap_exits in you gen_servers. >> > > This is very interesting, essentially because I am doing the same thing as > Bernard; but, for what I have understood from the supervisor documentation, > if I want a child to be supervised, it has to implement one of the four > behaviours, doesn't it? So, if my child doesn't do that (it is simply a > process which executes a single function in loop), how can I attach it to a > supervisor? I haven't used this, but take a look at: http://erldocs.com/R13B04/stdlib/gen_server.html?search=gen_server&i=17#enter_loop/5 Garrett From bartlomiej@REDACTED Wed May 5 21:29:35 2010 From: bartlomiej@REDACTED (=?utf-8?Q?Bart=C5=82omiej_Puzo=C5=84?=) Date: Wed, 5 May 2010 19:29:35 +0000 (GMT) Subject: [erlang-questions] Designing supervision trees In-Reply-To: Message-ID: <679855289.303011273087775395.JavaMail.root@zimbra> > but, for what I have understood from the supervisor > documentation, if I want a child to be supervised, it has > to implement one of the > four behaviours, doesn't it? So, if my child doesn't do that (it is > simply a process which executes a single function in loop), how can I attach > it to a supervisor? In order to put your process into a supervision tree, you have to make it compliant with the OTP design principles. For more details, check: http://www3.erlang.org/doc/design_principles/spec_proc.html Saludos, -- Bart?omiej Puzo? Erlang Solutions --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From fritchie@REDACTED Wed May 5 21:39:45 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 05 May 2010 14:39:45 -0500 Subject: [erlang-questions] Designing supervision trees In-Reply-To: Message of "Wed, 05 May 2010 13:45:20 CDT." Message-ID: <34880.1273088385@snookles.snookles.com> On Wed, May 5, 2010 at 6:01 AM, Alessandro Sivieri wrote: >> [...] but, for what I have understood from the supervisor >> documentation, if I want a child to be supervised, it has to >> implement one of the four behaviours, doesn't it? So, if my child >> doesn't do that (it is simply a process which executes a single >> function in loop), how can I attach it to a supervisor? Garrett Smith wrote: gs> I haven't used this, but take a look at: gs> http://erldocs.com/R13B04/stdlib/gen_server.html?search=gen_server&i=17#enter_loop/5 Garrett et al., there's an OTP stdlib module specifically for that kind of situation, see 'supervisor_bridge'. Not-compiled-or-tested code follows: -module(my_bridge). -export([go_daddy/3, init/1, terminate/2]). go_daddy(A, B, C) -> supervisor_bridge:start_link(?MODULE, [A, B, C]). init(ArgList) -> Pid = proc_lib:spawn_link(not_otp_compliant, start_er_up, ArgList), {ok, Pid, Pid}. terminate(_Reason, Pid) -> exit(Pid, kill). %% Or perhaps send a message to pid asking it to stop? %% Whatever fits the need.... Stepping back to the general question of using supervisors, I recommend a bit of experimentation. It usually isn't *too* hard to modify your supervisor hierarchy with a slightly different shape. To mangle an ancient proverb, "An experiment is worth a thousand words." Oh, and make certain that the 'sasl' application is running so you get some decent event logging when major processes are killed & restarted. Then use the "Kill" button on the 'appmon' GUI to see what happens when you kill certain processes. Or use kill/2 to kill them, if the GUI isn't convenient. Then you'll see exactly how the supervisors react to a failure just about anywhere in your hierarchy. -Scott From prikrutil@REDACTED Wed May 5 22:52:31 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Thu, 6 May 2010 00:52:31 +0400 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: Hello. > but it's NOT a hashmap because it's NOT mutable, the keys > may only be atoms, and it's intended for fast matching of > groups of at most low dozens of entries, being a replacement > for -record, not dictionaries. Actually I would prefer not to be limited by "keys are only atoms" rule (but still having pattern-matching functionality). For example, dictionary where keys can be strings is superior for representing JSON documents, that, say, can be fetched from MongoDB and have a lot of long strings as keys. -- Sergey Samokhin From alisdairsullivan@REDACTED Wed May 5 23:03:59 2010 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Wed, 5 May 2010 14:03:59 -0700 (PDT) Subject: [erlang-questions] Binary Matching Restrictions In-Reply-To: References: <796AEBFE-728D-45BE-88CD-CF3CE7AE903B@yahoo.ca> Message-ID: <215812.68856.qm@web63602.mail.re1.yahoo.com> >> Curious as to why functions had to end with a tail call to a function that performs a binary >> match instruction in order for optimizations to be applied, > On what do you base that claim? I misspoke, I meant specifically in the case of a recursive function. My question is why the optimization is not applied in the following case (some clauses omited): f(<>) -> ??? ... ??? g(Rest, ...). g(Rest, ...) -> ??? ... ??? f(Rest). The documentation of the beam_bsm.erl module indicates the optimization should be applied in this case but bin_opt_info?shows it is not. From bernie@REDACTED Thu May 6 02:11:44 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 06 May 2010 10:11:44 +1000 Subject: [erlang-questions] Designing supervision trees In-Reply-To: <34880.1273088385@snookles.snookles.com> References: <34880.1273088385@snookles.snookles.com> Message-ID: <4BE20940.60606@m5net.com> Hi Scott et al, On 06/05/10 05:39, Scott Lystig Fritchie wrote: > Stepping back to the general question of using supervisors, I recommend > a bit of experimentation. It usually isn't *too* hard to modify your > supervisor hierarchy with a slightly different shape. > It's already gone through several shapes as I've experimented :) I was more looking for guidence on the "erlang way" to think about the problem than a specific concrete solution. The problem is not that I can't find a bunch of seemingly valid ways to do it, but that I'm afraid there's some considerations I haven't taken into account. > Oh, and make certain that the 'sasl' application is running so you get > some decent event logging when major processes are killed& restarted. > Good advice. We already have that, plus our own logging system built upon it which allows us to split logs by application and also have separate specific logs for "extra bad" stuff like crashes - makes them easier to pick up amongst the less critical noise of debugging info. > Then use the "Kill" button on the 'appmon' GUI to see what happens when > you kill certain processes. Or use kill/2 to kill them, if the GUI > isn't convenient. Then you'll see exactly how the supervisors react to > a failure just about anywhere in your hierarchy. > Appmon's 'kill' button was exactly how I was testing it :) Good to know my approach is a sound one - thanks. Thanks very much for everyone's suggestions. In particular the supervisor_bridge behaviour - hadn't seen that before and hadn't quite noticed that not all my supervised processes were gen_somethings. I think I'll just go and fix that now... B From toby@REDACTED Thu May 6 02:11:46 2010 From: toby@REDACTED (Toby Thain) Date: Thu, 6 May 2010 10:11:46 +1000 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <0CCCA302-CA93-4960-BBCC-6C72D45F545F@telegraphics.com.au> On 5-May-10, at 11:08 PM, Hynek Vychodil wrote: > On Wed, May 5, 2010 at 2:17 PM, Tim Fletcher > wrote: >>> While your examples might work if the value is a simple constant or >>> variable allowing any legal general expression will cause problems. >>> How would you interpret <{erlang:list_to_integer("123")}> ? Note >>> that >>> white space is usually ignored so making white space significant >>> here >>> would be creating a special case for just this. >> >> You're right, that's definitely not workable. >> >>> This was the main reason that = is used in the record syntax. >>> While ~ >>> may look strange using : will cause problems. How about =? Though >>> this >>> may force you to parenthesize some expressions. >> >> I think = is fine aesthetically, but Richard notes in his frames >> proposal that it could also be confused with existing syntax that >> uses >> =. Adopting the Ruby style "arrows" would be another option: >> >> <{author=>"Mike", erlang=>list_to_integer("123")}> >> >> Looks clear enough, but might be a bit too verbose for typing out? > > Just to be correct it is not Ruby style but Perl style and it was > found brief enough even in Perl sense. Also PHP. --Toby > >> >> Tim >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From tony.arcieri@REDACTED Thu May 6 03:18:13 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Wed, 5 May 2010 19:18:13 -0600 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: On Mon, May 3, 2010 at 5:10 PM, Richard O'Keefe wrote: > My proposal for 'frames' has been sitting around for years. > but it's NOT a hashmap because it's NOT mutable I'm kind of confused as to what you're saying here. Dicts aren't mutable either. However, you can produce a new dict from an old dict and a key/keys to add, just like you can produce a new record which is created from an old record and a field/multiple fields to change. When you say "frames" wouldn't be mutable, how does that differ from dicts (or for that matter, records) -- Tony Arcieri Medioh! A Kudelski Brand From ok@REDACTED Thu May 6 04:08:26 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:08:26 +1200 Subject: [erlang-questions] strip end off of string In-Reply-To: References: Message-ID: On May 5, 2010, at 4:54 AM, Wes James wrote: > I see that fist ++ rest below will match the first part of the string > and the put the rest in "Rest": The compiler can turn [E1,E2,E3]++Rest into [E1|[E2|[E3|Rest]]], either in an expression context or a pattern context. In the same way, it can turn "ABC"++Rest into [$A|[$B|[$C|Rest]]] because "ABC" _is_ [$A,$B,$C]. > t2(First ++ "\">") -> > io:format("~s~n", [First]). This, however, cannot be so transformed. If you understand how lists work in Erlang (they are singly linked, with no stored length), you will understand why it _obviously_ can't be so transformed. That doesn't mean that a compiler couldn't support this construct. It would require special code (which "ABC"++_ does not), and above all it requires building a whole new copy of First at run time during pattern matching, while pattern matching is all about taking things apart. Binaries DO store a length, so something like <">> would be imaginable. If you need to peel stuff off the end of a list, the simplest way to do it is to reverse the list: t2(String) -> t2a(lists:reverse(String)). t2a(">\"" ++ Reversed) -> All_But_End = lists:reverse(Reversed), io:format("~s~n", [All_But_End]). (By the way, "First" STRONGLY suggests "first element", which does not apply here. Prefix, or All_But_Last_Two, or Front, or something which doesn't suggest an element, would be better.) next thing people would be asking for Front++"ABC"++Back, which doesn't have From ok@REDACTED Thu May 6 04:19:55 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:19:55 +1200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> Message-ID: <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> On May 5, 2010, at 10:02 AM, Tim Fletcher wrote: >> My proposal for 'frames' has been sitting around for years. >> I would write that as >> <{ author ~ "Mike" >> , text ~ "My first blog post!" >> , tags ~ ["mongodb","python","pymongo"] >> , date ~ erlang:now() >> }> > > As much as I dislike the double bracketing I can see the necessity. You must hate the <<...>> used for binaries, then. There just aren't any ASCII brackets left. The double guillemets ?? are available in Latin 1, but are crying out to be used for binaries. I've searched through Unicode, but apart from lunate brackets and tortoiseshell brackets, there's not much that doesn't look too much like other things. > Is > there any reason why using the more common choice of ":" to delimit > the keys and values would be a problem (instead of "~")? Having too many meanings for the same symbol is too confusing and error-prone. I didn't use "=" because that already means "match". I didn't use "->" because that already means "clause". And I didn't use ":" because that means "module prefix". And it doesn't just mean "module prefix", it normally has an atom on the left. The tilde, on the other hand, has no other meaning in Erlang. Look at a line containing fred:mary(27) and it looks like a function call. Look at a line containing fred ~ mary(27), and there is nothing else it could possibly be but a maplet. Nor is the use of "~" an innovation; much work at Xerox PARC used it. From ok@REDACTED Thu May 6 04:35:19 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:35:19 +1200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <007FD405-2EA5-4B49-A393-6C60D5DC45C6@cs.otago.ac.nz> On May 5, 2010, at 7:32 PM, Tim Fletcher wrote: > Yes, if the values are atoms and you don't put spaces before/after the > semicolon. Whilst that would be similar to a mod:fun call, there are > no brackets, so technically there's no ambiguity. Currently that usage > of the semicolon is illegal syntax, so presumably it could be made to > work. What semicolon? ":" is a colon. Erlang is not Javascript, so there is no compelling reason why it should *look* like Javascript. There are things that are technically unambiguous in a programming language, but that's very far from being a sufficient criterion for usability. For example, in Fortran 66 there were "Hollerith literals" where you wrote a string as the number of characters followed by those characters, e.g., 15H.NOT.REAL(CODE) for the string ".NOT.REAL(CODE)". The *compiler* never had any trouble with this, but *people* had trouble getting the number right and far worse trouble reading the result, which is why this feature isn't in any recent Fortran standard. (.NOT. is a Fortran logical negation operator and REAL is a Fortran intrinsic function.) I'm not sure what brackets [] have to do with it; if you mean there are no parentheses, there's not the slightest reason why there should not be: label ~ func(args) is perfectly possible and indeed in my experiments with using the notation I have found a maplet with a function call to be quite common. You have to consider not just what the *compiler* can process, but - what *people* can rapidly distinguish, - what can happen as a result of cut-and-paste editing, - what crude approximate parsers like syntax colouring editors might have trouble with - what's going to happen when people grep for things - the fact that spaces are not generally obligatory in Erlang, so just saying "you ought to write label: value" is no guarantee that it will be done that way From ok@REDACTED Thu May 6 04:43:17 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:43:17 +1200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <1273035294.5047.0.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <7774EF20-63A3-41B6-8535-23022AAFFAF6@cs.otago.ac.nz> On May 5, 2010, at 10:20 PM, Robert Virding wrote: > This was the main reason that = is used in the record syntax. While ~ > may look strange using : will cause problems. How about =? Though this > may force you to parenthesize some expressions. That would certainly ease the transition from records to frames. Ease of transition was a big feature of Joe Armstrong's proposed syntax. When I tried rewriting existing code to use frames instead of records, I actually found it helpful to be using a *different* symbol so that I could look at a multiline pattern or expression and tell at a glance whether I had converted it or not. Why do we find it easier to argue about the colour of the paint for the Ark than to climb aboard? From ok@REDACTED Thu May 6 04:47:04 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:47:04 +1200 Subject: [erlang-questions] closure on function argument In-Reply-To: References: <4BDC7512.60604@tmit.bme.hu> Message-ID: <792117E2-F3C6-4373-B031-181B14AE8687@cs.otago.ac.nz> On May 5, 2010, at 10:36 PM, Robert Virding wrote: > Yes, I know. The reason is rather simple in that there were different > influences at the times when these things were implemented. So > originally we were influenced by Prolog which resulted in unscoped > variables and at that time Erlang had very few "functional" > constructs. Later when we started adding more functional constructs, > funs, comprehensions and higher-order functions, we were influenced by > more "standard" functional languages and used their scoping rules. "Influences" is one thing, "inconsistency" is another. I've played with two alternative syntaxes for Erlang, one of them about as close to Haskell as I could get, with Haskellish scope rules, and one pretty close to existing Erlang, with Erlangish scope rules. Both are easier to use than the current mixup. > Regrettable I admit, > > Robert > > On 3 May 2010 01:33, Richard O'Keefe wrote: >> >> On May 2, 2010, at 8:56 AM, Robert Virding wrote: >> >>> The variables in the arguments of a fun, Y in your example, are >>> always >>> "fresh" variables and never imported from the funs environment. >> >> For me personally, this is perhaps THE most confusing thing >> about Erlang. >> >> I have *no* problem with the same rule in ML or Haskell because they >> apply it *consistently*: every pattern everywhere introduces fresh >> variables, it is never ever possible in those languages for a >> variable occurrence in a pattern to be an applied occurrence; all >> pattern variables are binding occurrences. This applies to OCaml and >> Clean as well. >> >> But Erlang does NOT apply this rule consistently. >> Variables in a pattern of a 'case' are *not* fresh, they *may* be >> applied occurrences. And even in a fun, *some* variable occurrences >> may be applied ones: fun (Y,Y) -> ... end. >> >> It's especially confusing because variables in fun bodies and guards >> are imported as usual, so they are clearly in scope. >> >> It would be *so* nice if ALL variables in Erlang followed exactly >> the same rules. >> >> Here's a little example. Suppose someone has >> case E of P -> B end >> but doesn't want any variable bindings in B to be exported. >> The obvious thing to do is to rewrite it as >> (fun (P) -> B end)(E) >> but because Erlang is inconsistent here, that does not work. >> (You can patch around it with fun, case, and a new variable, >> but that has problems of its own.) >> >> I have never been able to see any merit in this rule; its effect >> on me is to make it harder to write funs because I always have to >> watch out for cases where the expected thing won't happen and it >> has to be patched around. >> >> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ok@REDACTED Thu May 6 04:47:51 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:47:51 +1200 Subject: [erlang-questions] closure on function argument In-Reply-To: References: <4BDC7512.60604@tmit.bme.hu> Message-ID: <82F645D1-63C0-4DF2-87E9-B3B5F44D721B@cs.otago.ac.nz> Sorry, I was composing a reply and decided to delete it unsent. I am in a bit of a hurry, and hit the wrong button. From ok@REDACTED Thu May 6 04:58:06 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 6 May 2010 14:58:06 +1200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> On May 6, 2010, at 8:52 AM, Sergey Samokhin wrote: > Actually I would prefer not to be limited by "keys are only atoms" > rule (but still having pattern-matching functionality). Remember what the frames proposal is FOR. It is NOT a "dictionaries" or "hash tables" proposal or anything even remotely like that. It's a "clean replacement for the -record kluge" proposal. To that end, _fast_ construction and matching and _low_ memory requirements are "must haves". The report explains this in some detail. There could well be some kind of pattern matching support for dictionaries. However, that's a DIFFERENT purpose with DIFFERENT tradeoffs. > For example, > dictionary where keys can be strings is superior for representing JSON > documents, that, say, can be fetched from MongoDB and have a lot of > long strings as keys. The frames proposal is NOT about JSON. It's about replacing Erlang -records. There are proposals on the table (again, one from Joe Armstrong, and one from me that goes into some detail about implementation) for allowing large numbers of possibly large atoms -- that one is an EEP. If/when that's implemented, frames would become usable for JSON, representing those strings as dynamic atoms. But that's pretty much a lucky accident. You might want to think about reducing the size of those JSON keys in order to reduce transport costs. From corticalcomputer@REDACTED Thu May 6 05:16:53 2010 From: corticalcomputer@REDACTED (G.S.) Date: Wed, 5 May 2010 20:16:53 -0700 Subject: Anyone know of a Physics simulation in Erlang? Message-ID: Hello all, Is anyone familiar with a Physics simulation program written in Erlang? Something that simulates 3d space and at least solid body physics?\ Regards, -Gene From bgustavsson@REDACTED Thu May 6 08:13:23 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 6 May 2010 08:13:23 +0200 Subject: [erlang-questions] Binary Matching Restrictions In-Reply-To: <215812.68856.qm@web63602.mail.re1.yahoo.com> References: <796AEBFE-728D-45BE-88CD-CF3CE7AE903B@yahoo.ca> <215812.68856.qm@web63602.mail.re1.yahoo.com> Message-ID: On Wed, May 5, 2010 at 11:03 PM, alisdair sullivan wrote: > I misspoke, I meant specifically in the case of a recursive function. My question is why the optimization is not applied in the following case (some clauses omited): > > f(<>) -> > ??? ... > ??? g(Rest, ...). > > g(Rest, ...) -> > ??? ... > ??? f(Rest). > > The documentation of the beam_bsm.erl module indicates the optimization should be applied in this case but bin_opt_info?shows it is not. beam_bsm mostly only analyzes code within a single function, thought the comments do not explicitly say so. To get your code optimized, you will need to move the code for the g function into the f/1 function. Alternately, the following should also also work: f(<>) -> ... g(...), f(Rest). g(...) -> ... -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From cowboymathu@REDACTED Thu May 6 09:58:07 2010 From: cowboymathu@REDACTED (=?UTF-8?B?TUF0aHV2YXRoYW5hbiBNT3UgfHwgIOCuruCupOCvgeCuteCupOCuqeCuqeCvjSDgrq7gr4w=?=) Date: Thu, 6 May 2010 13:28:07 +0530 Subject: http:request timout issue Message-ID: Hi All, I do asynchronous http:request like this case catch http:request(get,{Login_url,[]}, [{timeout, 360000}], [{sync, false}]) of {ok,RequestId} -> receive {http, {RequestId, Result}} -> case Result of {error,etimedout} -> do_something; {error,econnrefused} -> do_something; ------------------------------- ------------------------------- Erlang release R13B01 Normally I receive {error, etimeout} after 360 seconds but this time I received {error, timeout} after 2 seconds. Note that it is {error, timeout} not {error, etimeout}. Can someone please explain this case. Thanks, -- Mathuvathanan Mou. From steven.charles.davis@REDACTED Thu May 6 10:38:45 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 6 May 2010 01:38:45 -0700 (PDT) Subject: Anyone know of a Physics simulation in Erlang? In-Reply-To: References: Message-ID: <21a6218a-46d3-4dba-994a-8045f4f3b580@p2g2000yqh.googlegroups.com> I remembered this blog from a couple of years ago which recorded the author's efforts with 3d simulations in erlang during 2008... http://horizontalgames.blogspot.com/ ...from what I can see from a post on it last April the author declared willingness to share the source code of his experiments. /s On May 5, 10:16?pm, "G.S." wrote: > Hello all, > > Is anyone familiar with a Physics simulation program written in Erlang? > Something that simulates 3d space and at least solid body physics?\ > > Regards, > -Gene > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From rapsey@REDACTED Thu May 6 10:44:36 2010 From: rapsey@REDACTED (Rapsey) Date: Thu, 6 May 2010 10:44:36 +0200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: > > Actually I would prefer not to be limited by "keys are only atoms" > rule (but still having pattern-matching functionality). For example, > dictionary where keys can be strings is superior for representing JSON > documents, that, say, can be fetched from MongoDB and have a lot of > long strings as keys. > > -- > Sergey Samokhin > > I hope you realize keys in mongodb are a part of the document that gets saved as well. So you're wasting a lot of space if you're using long key names in documents. Sergej From johanmon@REDACTED Thu May 6 11:03:19 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 06 May 2010 11:03:19 +0200 Subject: multicore performance fine grained concurrency Message-ID: Hi, I'm running some benchmarks on a AMD Phenom X4/Kubuntu 9.10/R13B04 and have problems with the performance when using smp. The benchmark is a replicated store where each replicated process has an ets table. The states are kept synchronized by an underlying layer that creates an additional group process for each replicated process (this will change so don't worry to much about how to reimplement it). Having only one replicated process (not much of replication but I want to see the overhead), an application layer will use a function interface to send it a message. The replicated process then forwards the message to the group layer process. The group layer process does not know it is the leader (again, inefficient but thats not the point), will send it self a message. When this is received it will forward it to the replicated process who will do the update of the ets table. Note these are all asynchronous messages i.e. the application process nor the replicated process are suspended waiting for a reply. Each application layer function call will thus generate four messages: Appl -> Repl -> Group -> Group -> Repl In a benchmark wit 24000 of these operations (plus one asynchronous message to make sure that all previous messages have been handled) I have the following figures: smp 4:4 -> 126 ms smp 2:2 -> 143 ms smp disabled -> 65 ms :-( If this is the normal behavior for programs with fine grained concurrency? On the good side is that the whole setup is to take advantage of the replicated state to do read operations local without involving the group layer. With a ratio of 1/32 (one write in 32 operatons) the overhead is not that big. My initial figures with increaing number of replicas gave the following nice figures (R is the number of replicas), the 24000 operations are divided equally between R application processes): smp 4:4 / R = 1 -> 44 ms smp 4:4 / R = 2 -> 32 ms smp 4:4 / R = 3 -> 29 ms smp 4:4 / R = 4 -> 28 ms smp 4:4 / R = 6 -> 32 ms smp 4:4 / R = 8 -> 36 ms This looks great until you compare with smp disabled smp dis / R = 1 -> 25 ms smp dis / R = 2 -> 26 ms smp dis / R = 3 -> 27 ms smp dis / R = 4 -> 27 ms So my four cores can almost beat one of the cores :-) I have not tried on an Intel platform, is there a difference? Should smp default to 1:1? Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From prikrutil@REDACTED Thu May 6 11:44:18 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Thu, 6 May 2010 13:44:18 +0400 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: I see that there are four questions we are discussing here. So our decision tree would be: #1. Do we need a special syntax for dictionaries? Thing that lead me to wish a special syntax for dictionaries was JSON/YAML documents, where keys are strings (or even integers stored as strings). JSON is a good example, so I will use it to illustrate where dictionaries can be useful. When to use: * You don't know for sure *how many* keys there will be. JSON can be sent to you through RESTful interface with a lot of fake keys, and you don't want to store them as atoms in the global atom storage. * You don't know for sure *what* keys there will be. Sometimes key you have is a combination of different nested keys. In Redis database it's common to use keys like "user/name". Some subkeys can be integers. There can be something behind the string representation, so you may want to parse the key. Atoms aren't strings, so what you can do on it is limited. * You want to have pattern matching Manipulation such a JSON-documents without pattern matching is a pain. With dictionaries you shouldn't be restricted to atoms as keys. If your keys are known for your program, you probably want something more suitable for structures like frames/structs. #1.1 If we need syntax for dictionaries, what syntax to use? #2. Do we need first class replacement (frames/structs) for records? Yes, because records have some well known runtime restrictions. #2.1 If we need frames/structs, what syntax to use? So dictionaries are more "dynamic", when frames/struct are more "static". > Remember what the frames proposal is FOR. > It is NOT a "dictionaries" or "hash tables" proposal or anything > even remotely like that. Actually my previous message wasn't about "let's use term() for keys in frames". I really would like to have both of them implemented. I wrote the first message because I felt the lack of syntax for dictionaries. > It's a "clean replacement for the -record kluge" proposal. I would be happy to have first-class replacement for records in Erlang. > To that end, _fast_ construction and matching and _low_ memory > requirements are "must haves". ?The report explains this in some > detail. I couldn't find your report. Could you post the link here? > You might want to think about reducing the size of those JSON > keys in order to reduce transport costs. Actually, the size of my JSON keys is small. I'm just a bit afraid of using atoms, because atoms aren't garbage collected and atoms aren't that good for string processing (regular expressions, parsing, validation) just because atoms aren't strings. Related links: Abstract patterns, structs and frames: http://www.erlang.org/pipermail/erlang-questions/2009-February/041921.html Here's hoping we get frames or structs in 2009!: http://erlang.org/pipermail/erlang-questions/2009-January/040961.html Fix up records so that record are first-class citizens and not just syntax saccharine for tuples? http://www.erlang.org/pipermail/erlang-questions/2008-November/039978.html -- Sergey Samokhin From prikrutil@REDACTED Thu May 6 11:45:52 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Thu, 6 May 2010 13:45:52 +0400 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: > I hope you realize keys in mongodb are a part of the document that gets > saved as well. So you're wasting a lot of space if you're using long key > names in documents. I know about the drawbacks of using long keys, so I use short keys =) -- Sergey Samokhin From alessandro.sivieri@REDACTED Thu May 6 12:07:15 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Thu, 6 May 2010 12:07:15 +0200 Subject: [erlang-questions] Designing supervision trees In-Reply-To: <34880.1273088385@snookles.snookles.com> References: <34880.1273088385@snookles.snookles.com> Message-ID: 2010/5/5 Scott Lystig Fritchie > Garrett et al., there's an OTP stdlib module specifically for that kind > of situation, see 'supervisor_bridge'. How this is supposed to be used? I mean, I have seen your example, so I think there should be a supervisor_bridge process (under the main supervisor) dedicated to spawn non-otp children, and it should be separated from the current gen_server process; but then, the supervisor_bridge:init function documentation says that the Pid returned should be the main subsystem Pid, but if the bridge launches several processes, which is the main one? Or there should be a bridge for each subprocess? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From mayamatakeshi@REDACTED Thu May 6 14:12:35 2010 From: mayamatakeshi@REDACTED (mayamatakeshi) Date: Thu, 6 May 2010 21:12:35 +0900 Subject: [erlang-questions] gen_udp:send and concurrent access to the same Socket In-Reply-To: <1273049252.5047.14.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <1273049252.5047.14.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: On Wed, May 5, 2010 at 5:47 PM, Bengt Kleberg wrote: > Greetings, > > Any processes can send on a socket. Only the owner can receive on a > socket. > > >From a design point of view it would seem better to me with a single > process that does all the sending. If that is not possible given your > performance requirements (measure) you can send from all processes. > Yes. That's the thing I was anticipating; I got problems losing (inter-process) messages when using a single process to control a TCP socket so I thought the same would happen with gen_udp. Thanks. br, takeshi > > > bengt > > On Wed, 2010-05-05 at 10:31 +0200, mayamatakeshi wrote: > > Hello, > > I'm testing with gen_udp. > > I want to pass the same Socket to several different processes so that > they > > can use it to send messages to an external endpoint from time to time. > The > > processes don't care about responses for the messages they send. > > So, is it OK to call gen_udp:send/4 from more than one process passing > the > > same Socket? > > My doubt is if gen_udp:send/4 takes care of concurrent access or if I > should > > do it myself, sending the packet to be transmitted as a message to a > process > > that owns the Socket. > > > > br, > > takeshi > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mail@REDACTED Thu May 6 14:41:42 2010 From: mail@REDACTED (Tim Fletcher) Date: Thu, 6 May 2010 05:41:42 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> Message-ID: <7fa28f37-2ec9-4da4-96f8-91acf35476b0@p2g2000yqh.googlegroups.com> > You must hate the <<...>> used for binaries, then. I don't, so it's probably because <{ uses two *different* characters that I find it a little peculiar. > What semicolon? ":" is a colon. My mistake. I meant colon, and no idea why I typed semicolon. > I'm not sure what brackets [] have to do with it; if you > mean there are no parentheses, there's not the slightest > reason why there should not be: Parentheses. British usage of the word "brackets" usually means parentheses. Sorry for the additional confusion. What I meant was that <<{erlang:list_to_integer}>> can't be confused with a function call because there are no parentheses. But as Robert clearly explains, the key point is that the values *can* be function calls, which gives rise to ambiguity. That's the "problem" that I did not forsee with using colons that I was asking about in my first question. > Why do we find it easier to argue about the colour of the paint > for the Ark than to climb aboard? Just because syntax might be a relatively trivial issue does not mean that it can't be discussed and alternatives considered/compared. I don't have anything to add to the "more important" discussion on "should we add frames/structs?" because I think they are a sound idea in principle and would much prefer using them to records. It is also my opinion that using a tilde is not the best syntax to use for seperating the keys and values. Whilst it can't be confused with any existing syntax, I don't think it makes the meaning of the code very clear. Looking at <{foo ~ bar}> doesn't make me think of "a key/ name mapped to a value". It looks more like some binary operator. Tim From mail@REDACTED Thu May 6 14:42:59 2010 From: mail@REDACTED (Tim Fletcher) Date: Thu, 6 May 2010 05:42:59 -0700 (PDT) Subject: Special syntax for dictionaries In-Reply-To: References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: <6005a73e-8bec-42fe-b916-0a618bd6eb4a@h11g2000yqj.googlegroups.com> > I couldn't find your report. Could you post the link here? I've been reading this: http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf From comptekki@REDACTED Thu May 6 15:20:33 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 6 May 2010 07:20:33 -0600 Subject: [erlang-questions] strip end off of string In-Reply-To: References: Message-ID: On Wed, May 5, 2010 at 8:08 PM, Richard O'Keefe wrote: > Thanks for your comments on the above :) > t2(String) -> > ? t2a(lists:reverse(String)). > The thought just came to me. Does reverse actually reverse all the data in string and work on it or does it just work from the end going backward. It seems there would be a performance hit on long "String"s if reverse actually reversed all the items in to another string then worked on it. Do you know how reverse actually does this? again, thx for your comments on (is it called erlang matching??) thx, -wes From james.hague@REDACTED Thu May 6 15:32:58 2010 From: james.hague@REDACTED (James Hague) Date: Thu, 6 May 2010 08:32:58 -0500 Subject: dividing a project across directories Message-ID: I know how to modify the code loading path, so beam files in different directories can be loaded. Easy! But is there a way to break a project into multiple directories and still be able to use c(module) from the shell to compile them without specifying a path? And can I specify where the compiled code should go, instead of always being written to the current directory? Up until now, I've always just tossed all my erl and beam files in one directory per project, and that's that. From mononcqc@REDACTED Thu May 6 15:37:35 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 6 May 2010 09:37:35 -0400 Subject: [erlang-questions] dividing a project across directories In-Reply-To: References: Message-ID: I've usually used an Emakefile for that and used the make module to do it. It will only re-compile the modules that changed and you can decide where they will go or if you want to load them when they're compiled. I'm not sure if there's a better way, but that was enough for me to stop looking further. On Thu, May 6, 2010 at 9:32 AM, James Hague wrote: > I know how to modify the code loading path, so beam files in different > directories can be loaded. Easy! > > But is there a way to break a project into multiple directories and > still be able to use c(module) from the shell to compile them without > specifying a path? And can I specify where the compiled code should > go, instead of always being written to the current directory? > > Up until now, I've always just tossed all my erl and beam files in one > directory per project, and that's that. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuncer.ayaz@REDACTED Thu May 6 18:37:08 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 6 May 2010 18:37:08 +0200 Subject: [erlang-questions] dividing a project across directories In-Reply-To: References: Message-ID: On Thu, May 6, 2010 at 3:32 PM, James Hague wrote: > I know how to modify the code loading path, so beam files in different > directories can be loaded. Easy! > > But is there a way to break a project into multiple directories and > still be able to use c(module) from the shell to compile them without > specifying a path? And can I specify where the compiled code should > go, instead of always being written to the current directory? > > Up until now, I've always just tossed all my erl and beam files in one > directory per project, and that's that. Hi James, have you tried packaged modules? It's experimental, unsupported and personally I do not want to encourage its use but it might solve your issue. http://www.erlang.se/publications/packages.html For specifying OutDir use c(module,{outdir,Dir}. From fritchie@REDACTED Thu May 6 19:05:43 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 06 May 2010 12:05:43 -0500 Subject: [erlang-questions] Designing supervision trees In-Reply-To: Message of "Thu, 06 May 2010 12:07:15 +0200." Message-ID: <99220.1273165543@snookles.snookles.com> Alessandro Sivieri wrote: sa> Or there should be a bridge for each subprocess? Correct. -Scott From dmurray@REDACTED Thu May 6 19:50:23 2010 From: dmurray@REDACTED (David N Murray) Date: Thu, 06 May 2010 13:50:23 -0400 (EDT) Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: Message-ID: On May 6, Johan Montelius scribed: > > > > smp 4:4 -> 126 ms > smp 2:2 -> 143 ms > smp disabled -> 65 ms > > :-( > I saw something similar using the Ring benchmark on both AMD (OpenBSD) and Intel (Vista) dual cores. Both cores get utilized in the 40-50% range with smp enabled. It takes 1/4 the time to run the benchmark with smp disabled as it does when smp is enabled. Takes advantage of two cores just fine if you run two OS processes with SMP disabled. Doesn't do so well SMP enabled. The ring benchmark just spawns and sends messages. smp 2:2 -> 18003 ms smp disabled -> 4867 ms 2 os processes -> ~5900 ms hth, Dave From corticalcomputer@REDACTED Thu May 6 20:06:06 2010 From: corticalcomputer@REDACTED (G.S.) Date: Thu, 6 May 2010 11:06:06 -0700 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: Message-ID: Hello, I've had the same problems in my distributed systems, as Joe put it in his book: "Big computations, small messages" is the way to leverage your distributed systems in Erlang. Also, creating separate nodes for every core, without smp in many cases let me operate at full speed. But as it stands something needs to be done about smp. Regards, -Gene On Thu, May 6, 2010 at 10:50 AM, David N Murray wrote: > On May 6, Johan Montelius scribed: > > > > > > > > > smp 4:4 -> 126 ms > > smp 2:2 -> 143 ms > > smp disabled -> 65 ms > > > > :-( > > > > I saw something similar using the Ring benchmark on both AMD (OpenBSD) and > Intel (Vista) dual cores. Both cores get utilized in the 40-50% range > with smp enabled. It takes 1/4 the time to run the benchmark with smp > disabled as it does when smp is enabled. Takes advantage of two cores just > fine if you run two OS processes with SMP disabled. Doesn't do so well > SMP enabled. The ring benchmark just spawns and sends messages. > > smp 2:2 -> 18003 ms > smp disabled -> 4867 ms > 2 os processes -> ~5900 ms > > hth, > Dave > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rvirding@REDACTED Thu May 6 20:26:31 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 6 May 2010 20:26:31 +0200 Subject: [erlang-questions] strip end off of string In-Reply-To: References: Message-ID: On 6 May 2010 15:20, Wes James wrote: > On Wed, May 5, 2010 at 8:08 PM, Richard O'Keefe wrote: >> > > > > Thanks for your comments on the above :) > >> t2(String) -> >> ? t2a(lists:reverse(String)). >> > > The thought just came to me. ?Does reverse actually reverse all the > data in string and work on it or does it just work from the end going > backward. ?It seems there would be a performance hit on long "String"s > if reverse actually reversed all the items in to another string then > worked on it. ?Do you know how reverse actually does this? You can code reverse in Erlang as: reverse(List) -> reverse(List, []). reverse([H|T], Acc) -> reverse(T, [H|Acc]); reverse([], Acc) -> Acc. Now lists:reverse/2 is coded in C but it works in the same way as this with the same result. Robert From thomasl_erlang@REDACTED Thu May 6 22:50:22 2010 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 6 May 2010 13:50:22 -0700 (PDT) Subject: Klarna/Kreditor + Sequoia Message-ID: <520948.95593.qm@web111416.mail.gq1.yahoo.com> I haven't seen it mentioned on the list, but Klarna/Kreditor (who do quite a bit of erlang) just got an investment by Sequoia Capital and are getting Michael Moritz on the board of directors to boot. Could we enthusiasts at this point say erlang has arrived? :-) Congratulations, guys. Keep at it. http://www.klarna.se/en/press/258-sequoia-capital-investerar-i-klarna Best, Thomas From kenneth.lundin@REDACTED Thu May 6 23:52:57 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 6 May 2010 23:52:57 +0200 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: Message-ID: On Thu, May 6, 2010 at 8:06 PM, G.S. wrote: > Hello, > > I've had the same problems in my distributed systems, as Joe put it in his > book: "Big computations, small messages" is the way to leverage your > distributed systems in Erlang. > Also, creating separate nodes for every core, without smp in many cases let > me operate at full speed. > > But as it stands something needs to be done about smp. > On the other hand there are many examples of real applications that really run well in a single smp enabled VM on a multicore HW, I don't know if your particular system also is a benchmark thing but I think there is a clear pattern that benchmark systems tend to perform or scale bad. I think the reason for that mostly is that they are not doing much computations between the process creations, message passings , ets-accesses and so which all requires some kind of locking. This tesult in that the total amount of lock requests will be a large part of the total work. The SMP performance is improved in every release and it is mostly about reducing or removing the need for locking. We know quite a lot about which bottlenecks there are and will tick them off one by one. /Kenneth Erlang/OTP Ericsson > Regards, > -Gene > > On Thu, May 6, 2010 at 10:50 AM, David N Murray wrote: > >> On May 6, Johan Montelius scribed: >> >> > >> > >> > >> > smp 4:4 -> 126 ms >> > smp 2:2 -> 143 ms >> > smp disabled -> 65 ms >> > >> > :-( >> > >> >> I saw something similar using the Ring benchmark on both AMD (OpenBSD) and >> Intel (Vista) dual cores. ?Both cores get utilized in the 40-50% range >> with smp enabled. ?It takes 1/4 the time to run the benchmark with smp >> disabled as it does when smp is enabled. Takes advantage of two cores just >> fine if you run two OS processes with SMP disabled. ?Doesn't do so well >> SMP enabled. ?The ring benchmark just spawns and sends messages. >> >> smp 2:2 -> 18003 ms >> smp disabled -> 4867 ms >> 2 os processes -> ~5900 ms >> >> hth, >> Dave >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From ok@REDACTED Fri May 7 00:45:49 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 7 May 2010 10:45:49 +1200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: Message-ID: <9AA69409-192C-4940-9BE0-12C806BE2137@cs.otago.ac.nz> On May 6, 2010, at 1:18 PM, Tony Arcieri wrote: > On Mon, May 3, 2010 at 5:10 PM, Richard O'Keefe > wrote: > My proposal for 'frames' has been sitting around for years. > but it's NOT a hashmap because it's NOT mutable > > I'm kind of confused as to what you're saying here. Dicts aren't > mutable either. Did I say they were? > > When you say "frames" wouldn't be mutable, how does that differ from > dicts (or for that matter, records) I nowhere said that frames differed from either dicts or records *in this respect*. I just said they weren't hashmaps. java.utils.HashMap is a well known Java class -- indeed, that is where the "word" hashmap comes from, before Java's class, people talked about hash tables or tables or hashes or dictionaries mostly. And hashmaps support put() and remove() mutation operations. If it comes to that, frames aren't _any_ kind of hash table, as in order to achieve the best space efficiency I could think of they do not use any form of hashing at all. This is another way in which they differ from dicts: you aren't really _expected_ to access one field at a time, you're expected to access _several_ fields. The report has one extended example. It's not the only one I've done. A manual optimisation I found often useful was to replace f(..., R, ...) -> ... use R's f ... ... use R's g ... ... use R's h ... by f(..., R = <{f ~ F, g ~ G, h ~ H}>, ...) -> ... use F ... ... use G ... ... use H ... Indeed, I did at one time think of allowing (frame-valued expression) ~ field with the obvious meaning, but dropped it in order to encourage this more efficient style. Of course a compiler can perfectly well gather all the field accesses on a path and put them into an earlier match, EXCEPT that this changes the failure behaviour of the program: if R doesn't _have_ an h field the upper example will crash when you try to get it, but the lower one will crash in the function's head, which is, other things being equal, a better thing to do. It also meant that I didn't have to give two related-but-confusingly-different readings to ~ . From rvirding@REDACTED Fri May 7 02:53:31 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 7 May 2010 02:53:31 +0200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: As has been mentioned we are actually discussing the syntax of two separate things here: frames/structs and dicts/"hashmaps". Frames/structs are an extension/replacement/improvement on records. Both give you a structure with named fields but where records are compile time and static not Erlangy, structs/frames are runtime and dynamic and much more in keeping with Erlang. But they are not an alternative to dicts, or at least they were not envisioned to be that. One fear I have always had with frames/structs (honestly I have :-) ) is that people would start using them as an alternative to dicts for which they were not intended, though I was thinking more of the problems of implementing them. That being said I am still all for them and will gladly take a discussion on syntax and properties. I will even accept the ~. I personally don't think that we need or should have a special syntax for dicts, it would in one sense defeat the purpose of them. My basic principle has been that there is no "best" implementation of a dict so by having a number of different implementations with the same interface so the could more easily test and choose. I will soon (real soon now) be adding a tree dict. Another reason for not wanting a special syntax for dicts/hashmaps is that I would like to avoid adding more special syntax where it isn't absolutely necessary and I don't feel that these really need it. Robert From steven.charles.davis@REDACTED Fri May 7 03:08:51 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 6 May 2010 18:08:51 -0700 (PDT) Subject: dividing a project across directories In-Reply-To: References: Message-ID: The following wildcards will work in Emakefiles: {"src/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. {"src/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. I think that's what you're asking for! /s From dujinfang@REDACTED Fri May 7 04:31:32 2010 From: dujinfang@REDACTED (Seven Du) Date: Fri, 7 May 2010 10:31:32 +0800 Subject: How to tell epmd to release a name rigsitered but not valid any more? In-Reply-To: <23f91031003032336v4d9bea58vc506210697af61ff@mail.gmail.com> References: <23f91031003032336v4d9bea58vc506210697af61ff@mail.gmail.com> Message-ID: Was there anyone but me has this problem? I have met quite a few times. $ which erl /usr/local/bin/erl $ erl -v Erlang R13B02 (erts-5.7.3) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.3 (abort with ^G) $ which epmd /usr/local/bin/epmd $ epmd -names epmd: up and running on port 4369 with data: name freeswitch at port 8031 name mos at port 61573 name queue at port 58107 name queue1 at port 16557 name voiceEvents at port 57340 For some reason I stoped queue, and it can never start claiming name already there, so I started the same application with another name queue1. As you can see queue1 (16557) is there but queue(58107) isn't. In my understanding, it should release the port if the socket broken. Or at least there's a way to release the port without killing the whole epmd to avoid affect other applications running on the same server. I haven't read the source code, however, I would like to make a patch if it is confirmed a bug and no one else would like to fix. Thanks. app@REDACTED:~/erlang_apps/queue$ netstat -an|grep 16557 tcp 0 0 0.0.0.0:16557 0.0.0.0:* LISTEN $ netstat -an|grep 58107 $ $ netstat -an|grep 8031 tcp 0 0 10.20.208.8:8031 0.0.0.0:* LISTEN tcp 0 0 10.20.208.8:8031 10.20.208.8:8984 ESTABLISHED tcp 0 0 10.20.208.8:32465 10.20.208.8:8031 ESTABLISHED tcp 0 0 10.20.208.8:8984 10.20.208.8:8031 ESTABLISHED tcp 0 0 10.20.208.8:8031 10.20.208.8:32465 ESTABLISHED tcp 0 0 10.20.208.8:8031 10.20.208.8:30683 ESTABLISHED tcp 0 0 10.20.208.8:30683 10.20.208.8:8031 ESTABLISHED $ netstat -an|grep 4369 tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:4369 127.0.0.1:23526 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:57900 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:21512 TIME_WAIT tcp 0 0 127.0.0.1:30721 127.0.0.1:4369 ESTABLISHED tcp 0 0 127.0.0.1:23526 127.0.0.1:4369 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:5976 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:30721 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:42265 ESTABLISHED tcp 0 0 127.0.0.1:42265 127.0.0.1:4369 ESTABLISHED tcp 0 0 127.0.0.1:57900 127.0.0.1:4369 ESTABLISHED tcp 0 0 127.0.0.1:4369 127.0.0.1:21535 TIME_WAIT tcp 0 0 127.0.0.1:23947 127.0.0.1:4369 ESTABLISHED 2010/3/4 Seven Du : > Hi, > > It is noticed that sometime when an erlang app crashes the empd daemon > still hold the node name, then the crashed app cannot restart with the > same name. Though kill epmd solves this problem, it is not a good > practice when many erlang apps runs on the same server. > > epmd seems only has epmd -names to list all registered nodes, is there > a way to unregister a name from epmd? > > Thanks. > -- Blog: http://www.dujinfang.com Proj: http://www.freeswitch.org.cn From ok@REDACTED Fri May 7 07:03:19 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 7 May 2010 17:03:19 +1200 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: <3A6F3D19-2834-452A-A077-EF7F740CF571@cs.otago.ac.nz> On May 6, 2010, at 9:44 PM, Sergey Samokhin wrote: > * You don't know for sure *how many* keys there will be. > > JSON can be sent to you through RESTful interface with a lot of fake > keys, and you don't want to store them as atoms in the global atom > storage. We shouldn't have to care about that. There is a great gaping bleeding vulnerability at the heart of Erlang: it SHOULDN'T be possible to bring down a system by sending it a lot of distinct atoms, but it IS. It doesn't have to be that way. EEP 20, by someone whose initials look a lot like mine, says An idea from the Logix implementation of Flat Concurrent Prolog can be adapted to Erlang: invisibly to users there can be two implementations of 'atoms', fixing a major system integrity issue and removing the need to warp one's data structure design to code around it. Historically, many Prolog systems had a similar limitation. When I got a copy of Brand X Prolog for the Macintosh, about the 10th thing I tried was creating more atoms than its atom table could hold. Brand X Prolog not only crashed, it managed to wipe itself off the disc on the way to the grave, so I was never able to try any other tests. (Silly me, I had thought that since I wasn't trying to change anything on that floppy, it would not change.) A couple of years ago, Jan Wielemaker fixed SWI Prolog, so that it DOES garbage collect atoms, and it is now common practice to use atoms all over the place, in large numbers. This despite the fact that SWI Prolog also has (sadly pthread-like) concurrency. So "we need to use something other than atoms for JSON labels" is a *temporary* concession to a bug in the surviving Erlang implementation (GERL and Erlang-over-Scheme apparently being dead) which has less excuse for remaining every day. I don't think we should allow a bug that's overdue for fixing to warp the design of *future* language changes. > > * You don't know for sure *what* keys there will be. Then we are definitely talking about dictionaries. But if you don't know what the keys will be, you can't write them in patterns or expressions, so what good would special syntax be? > > Sometimes key you have is a combination of different nested keys. Nested keys should be handled by nested dictionaries each with a simple key. For example, in Java on my old machine, System.getProperties.list(System.out) shows me 48 keys (and their values) in an apparently random order. Suppose they were factorised file/... java/... user/... line/... os/... sun/... Each level would be easier to grasp, and groups of properties, like java/vm/... and os/... and user/... would actually make sense. And each level would be so small that simple linear search would work well. As a frame, we'd have <{ file ~ <{ encoding ~ <{ '.' ~ _1 , pkg ~ _2 }> , separator ~ _3 }> , java ~ ... , line ~ ... , os ~ ... , sun ~ ... , user ~ <{ country ~ _U , dir ~ _V , home ~ _W , language ~ _X , name ~ _Y , timezone ~ _Z }> }> > In > Redis database it's common to use keys like "user/name". Some subkeys > can be integers. There can be something behind the string > representation, so you may want to parse the key. Atoms aren't > strings, so what you can do on it is limited. Non-sequitur. There is nothing that can be done with strings that *couldn't* be done with atoms. For example, in Haskell, practically everything you might want to do with a list of characters (String) can be done with a Data.ByteString. In particular, there's no reason why you couldn't be allowed to concatenate atoms, slice out subatoms, and do regular expression matching on atoms. *IF* the global limited atom table weren't a scarce and vulnerable resource, it would make perfect sense to do these things. SWI Prolog even lets you read a term as an atom. (Makes me queasy, but it _works_.) > > * You want to have pattern matching > Manipulation such a JSON-documents without pattern matching is a > pain. I've done a fair bit of XML processing in Prolog, and for that matter, in C. (I have my own faster-than-expat parser and my own "Document Value Model" library. It's amazing what you can do with XML _conveniently_ in C.) JSON is really very like XML. They are *both* hard to process with or without patterns, for the same reason: they are only semistructured and there can be all sorts of extra cruft there. For example, I'm currently working with 100,000 US patents, marked up in XML. You would think "where is *the* ECLA code saying what this is about" would be a simple matter of fishing quickly in the meta-data, but it isn't, and for what I actually have to do to get a best-guess at the value, no plausible pattern language would be a substantial improvement on the actual code I have. There's an old statistician's saying: raw data are like raw potatoes, they're no use until they're cleaned. It applies to open semistructured data coming into a computer program as well: first thing you do is extract just the data you want and clean it up to make life as easy as possible for down-stream code. (Hint: the letters "JS" in "JSON" stand for what? That has what pattern matching facilities? Programmers using it complain about the difficulty of using JSON data how much?) > With dictionaries you shouldn't be restricted to atoms as keys. With dictionaries, you aren't. Never have been, never will be. > If > your keys are known for your program, you probably want something more > suitable for structures like frames/structs. > > #1.1 If we need syntax for dictionaries, what syntax to use? Something extensible. There can be more than one kind of dictionary. (As a trivial example: are keys matched using ==, or =:=, or something else, which might perhaps ignore alphabetic case?) > > #2. Do we need first class replacement (frames/structs) for records? > > Yes, because records have some well known runtime restrictions. > > #2.1 If we need frames/structs, what syntax to use? > > So dictionaries are more "dynamic", when frames/struct are more > "static". That's a fair characterisation. While it is always possible to add any key to any frame (getting a new frame), and while it is possible for a frame to have keys that the receiver was not expecting, the _creator_ of a frame must have a specific set of keys in mind. (Although there is a pattern-like syntax and ALSO a fully dynamic set of functions.) > Actually my previous message wasn't about "let's use term() for keys > in frames". I really would like to have both of them implemented. I > wrote the first message because I felt the lack of syntax for > dictionaries. I recommend writing a few modules *pretending* that such a syntax exists and see what it ends up looking like. You'll keep on changing your mind for a while, but eventually something will gell. > I couldn't find your report. Could you post the link here? Here it is again: www.cs.otago.ac.nz/staffpriv/ok/frames.pdf > Actually, the size of my JSON keys is small. I'm just a bit afraid of > using atoms, because atoms aren't garbage collected and atoms aren't > that good for string processing (regular expressions, parsing, > validation) just because atoms aren't strings. At the risk of sounding like "Poor Johnny One-Note" - atoms *should* be and *could* be garbage collected (Logix did it and SWI Prolog does it) - atoms *could* be processed by regular expressions as easily as character lists and binaries (they *are* in Smalltalk) Right now, you are right. Fixing the atom garbage collection problem ought to be near the top of the priority list, I think. Note: EEP 20 proposes implementing atoms two different ways: 'global' atoms that appear in source code and 'dynamic' ones. One reason for this is that no changes to the compiler or to its .beam files are needed, only VM changes. > > Related links: > > Abstract patterns, structs and frames: > http://www.erlang.org/pipermail/erlang-questions/2009-February/041921.html There *is* an EEP for abstract patterns, EEP 29. I did submit an EEP for frames but it was rejected as too large. Basically, http://www.otago.ac.nz/staffpriv/ok/frames.pdf *is* the EEP for frames, and has been since late 2003. From ok@REDACTED Fri May 7 07:30:55 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 7 May 2010 17:30:55 +1200 Subject: [erlang-questions] strip end off of string In-Reply-To: References: Message-ID: <77A1754B-3894-4641-A0B8-9D5E0787D946@cs.otago.ac.nz> On May 7, 2010, at 1:20 AM, Wes James wrote: > On Wed, May 5, 2010 at 8:08 PM, Richard O'Keefe > wrote: >> > > > > Thanks for your comments on the above :) > >> t2(String) -> >> t2a(lists:reverse(String)). >> > > The thought just came to me. Does reverse actually reverse all the > data Yes. > in string and work on it or does it just work from the end going > backward. It can't. Let's make three strings. X = "bit", % X : CONS, $a, ->X' % X' : CONS, $b, ->X" % X" : CONS, $c, [] Y = "or" ++ X, % Y : CONS, $o, ->Y' % Y' : CONS, $r, ->X Z = "rab" ++ X, % Z : CONS, $r, ->Z' % Z' : CONS, $a, ->Z" % Z" : CONS, $b, ->X The comment lines are supposed to show you an "assembly code level" view of the actual data structure. (Whether the CONS tags are in the *records* or on the *pointers* is immaterial. Normal C practice is to put the tags in the records; normal Lisp implementation practice is to put the tags on the pointers.) So Z is a pointer to a record holding an $r and a pointer to a record holding an $a and a pointer to a record holding a $b and a pointer which is the *same* pointer as X. The three records of "bit" are *shared* by both Y and Z. So if you "worked from the end going backward", (a) in the absence of back pointers, and they *are* absent, how could you do it at all? (b) when you get to the "b" of "bit", how do you tell whether to continue with the "r" of "orbit", or with the "b" of "rabbit", or to stop? If you have a BSD system, like a Mac, "man queue" will tell you about , which has SLIST, STAILQ, LIST, and TAILQ; the one that resembles Erlang is SLIST. Look in for the code. The bottom line is that lists can *only* be traversed from front to back, because there are *only* forward pointers. > It seems there would be a performance hit on long "String"s > if reverse actually reversed all the items in to another string then > worked on it. Do you know how reverse actually does this? Look at the code in lib/stdlib/src/lists.erl. lists:reverse/1 is now a built-in function, but it _used_ to be reverse(Xs) -> reverse(Xs, []). reverse([H|T], Y) -> reverse(T, [H|Y]); reverse([], X) -> X. If you want to tell whether a string ends with something, like "", for example, you could write ends_with_TD("") -> true; ends_with_TD([_|S]) -> ends_with_TD(S); ends_with_TD([]) -> false. One of the things I keep telling people is "STRINGS ARE WRONG". For many purposes, a list of tokens is superior to a string. (For most other purposes, some kind of tree is even better. I've had spectacular speedups in C from using trees of fragments instead of vectors of characters.) From ok@REDACTED Fri May 7 07:49:50 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 7 May 2010 17:49:50 +1200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: <7fa28f37-2ec9-4da4-96f8-91acf35476b0@p2g2000yqh.googlegroups.com> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> <7fa28f37-2ec9-4da4-96f8-91acf35476b0@p2g2000yqh.googlegroups.com> Message-ID: <92D2926F-8F9B-4986-AAA7-C83FD9F1EE3E@cs.otago.ac.nz> On May 7, 2010, at 12:41 AM, Tim Fletcher wrote: > > Parentheses. British usage of the word "brackets" usually means > parentheses. Sorry for the additional confusion. My dialect/chronolect is fairly close to British. However, I learned early on that talking about "brackets" was EXTREMELY error-prone. I try to redundantly say round parentheses square brackets curly braces angle brockets to be unambiguous. The OED (www.oed.com) says One of two marks of the form [ ] or ( ), and in mathematical use also {}, used for enclosing a word or number of words, a portion of a mathematical formula, or the like, so as to separate it from the >>> context; in typography, esp. applied to ?square brackets? >>> (formerly called crotchets), the ?round brackets? being >>> designated ?parentheses?. So the distinction between "brackets" and "parentheses" is not confined to (careful!) programmers. It looks as though people working in an area where it *matters* which kind of paired symbols you use distinguish between brackets and parentheses, whereas people working in an area where it doesn't (like it doesn't really matter whether you use (Freddo, 1633) or [Freddo, 1633] for citations) use brackets for both indifferently. For what it's worth, it was the >>European<< programming language Algol that taught me the importance of the distinction. > Just because syntax might be a relatively trivial issue does not mean > that it can't be discussed and alternatives considered/compared. I > don't have anything to add to the "more important" discussion on > "should we add frames/structs?" because I think they are a sound idea > in principle and would much prefer using them to records. There are none-the-less some important semantic questions. > > It is also my opinion that using a tilde is not the best syntax to use > for seperating the keys and values. Believe me, I've looked for something better, and there _isn't_ anything. > Whilst it can't be confused with > any existing syntax, I don't think it makes the meaning of the code > very clear. Looking at <{foo ~ bar}> doesn't make me think of "a key/ > name mapped to a value". It looks more like some binary operator. Well, if it comes to that, << makes me think of "very much less than" and "left shift". It *certainly* doesn't suggest "binary" to me. *Anything* you put between the label and the value of a maplet is going to look like some binary operator, in fact from a grammatical point of view it's going to BE a binary operator. The important thing is that it not look like some EXISTING binary operator in the language. And I repeat: this is a notation that HAS been used before. If it doesn't suggest "maplet" to you because you're not familiar with the relevant work at Xerox PARC, it's not the fault of the notation. There *is* a Unicode character which is perfect for the job. U+21A6 ? looks like exactly the maplet character. Back in 2003 I couldn't type Unicode characters into a mail message. In fact, I *still* can't, I have to use Insert|Special characters|... which is a pain in the neck. (It makes \mapsto a marvel of usability.) Realistically, one is often looking at <{ foo ~ SOME HAIRY EXPRESSION , bar ~ ANOTHER HAIRY EXPRESSION , ugh ~ YET A THIRD }> It's pretty obvious what's going on (as long as we don't use : or = or ->). Oh, the use of <{ ... }> was inspired by - these things are sets of pairs, so {...} is appropriate - I've seen overstruck < ( ... ) > used for string quotes From raimo+erlang-questions@REDACTED Fri May 7 09:27:48 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 7 May 2010 09:27:48 +0200 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: Message-ID: <20100507072748.GA14799@erix.ericsson.se> On Thu, May 06, 2010 at 01:50:23PM -0400, David N Murray wrote: > On May 6, Johan Montelius scribed: > > > > > > > > > smp 4:4 -> 126 ms > > smp 2:2 -> 143 ms > > smp disabled -> 65 ms > > > > :-( > > > > I saw something similar using the Ring benchmark on both AMD (OpenBSD) and > Intel (Vista) dual cores. Both cores get utilized in the 40-50% range > with smp enabled. It takes 1/4 the time to run the benchmark with smp > disabled as it does when smp is enabled. Takes advantage of two cores just > fine if you run two OS processes with SMP disabled. Doesn't do so well > SMP enabled. The ring benchmark just spawns and sends messages. There are many different reasons why SMP, especially SMP benchmarks (as Kenneth explained in another mail in this thread) performs poorly. OpenBSD still does not have native threads, so one OS process only runs on one CPU at the time. Threads are implemented as old style (green) threads within that process. The SMP emulator starts, probably with as many schedulers as there are CPUs, runs on both schedulers within one CPU thread, and the OS distributes that load over both CPUs. So max possible utilization will be 50% per CPU. Vista seems to be very eager to distribute the load over the CPUs, so execution jumps between them like crazy, which destroys the CPU memory cache for every jump, slowing down execution. Intel before i7/i5 has much less memory bandwidth, especially between cores, so the SMP emulator performs worse on them than on i7/i5. Our current best combo I guess is Linux (perhaps Solaris maybe FreeBSD) on Intel I7. And for the ring benchmark (single ring) is it not so that it sends a message in a ring so there is only one process at every instant that can execute. So all the SMP emulator can contribute is overhead, making this benchmark only measure how much overhead the SMP emulator has for pure message passing and process scheduling. The SMP emulator can never beat the non-SMP emulator for a single ring and it can only load one CPU no more than 100%. > > smp 2:2 -> 18003 ms > smp disabled -> 4867 ms > 2 os processes -> ~5900 ms > > hth, > Dave > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From schramm.ingo@REDACTED Fri May 7 10:24:44 2010 From: schramm.ingo@REDACTED (ingo.schramm) Date: Fri, 7 May 2010 01:24:44 -0700 (PDT) Subject: multicore performance fine grained concurrency In-Reply-To: <20100507072748.GA14799@erix.ericsson.se> References: <20100507072748.GA14799@erix.ericsson.se> Message-ID: <89dd8d05-95c1-477b-97da-f6689fdfc7ad@s29g2000yqd.googlegroups.com> During the last months I did a lot of benchmarking SMP, not to learn how Erlang SMP performs but rather to learn how to write code that scales well with Erlang SMP. I had different machines available, even 8 cores with hyperthreading resulting in 16 logical cores. What I found is that it is not *that* easy to write code that scales (almost) linearly, but it is possible. Bad scaling behaviour almost always uncovers serial bottlenecks in the code (like the ring, I guess). On the other hand, if you have code with a lot of serial paths you benefit from turning off SMP because the SMP scheduling has its own overhead. This is natural. On May 7, 9:27?am, Raimo Niskanen wrote: > On Thu, May 06, 2010 at 01:50:23PM -0400, David N Murray wrote: > > On May 6, Johan Montelius scribed: > > > > smp 4:4 -> 126 ms > > > smp 2:2 -> 143 ms > > > smp disabled -> 65 ms > > > > :-( > > > I saw something similar using the Ring benchmark on both AMD (OpenBSD) and > > Intel (Vista) dual cores. ?Both cores get utilized in the 40-50% range > > with smp enabled. ?It takes 1/4 the time to run the benchmark with smp > > disabled as it does when smp is enabled. Takes advantage of two cores just > > fine if you run two OS processes with SMP disabled. ?Doesn't do so well > > SMP enabled. ?The ring benchmark just spawns and sends messages. > > There are many different reasons why SMP, especially SMP benchmarks > (as Kenneth explained in another mail in this thread) performs poorly. > > OpenBSD still does not have native threads, so one OS process only runs on > one CPU at the time. Threads are implemented as old style (green) > threads within that process. The SMP emulator starts, probably with > as many schedulers as there are CPUs, runs on both schedulers > within one CPU thread, and the OS distributes that load over > both CPUs. So max possible utilization will be 50% per CPU. > > Vista seems to be very eager to distribute the load over the CPUs, > so execution jumps between them like crazy, which destroys the > CPU memory cache for every jump, slowing down execution. > > Intel before i7/i5 has much less memory bandwidth, especially > between cores, so the SMP emulator performs worse on them than > on i7/i5. > > Our current best combo I guess is Linux (perhaps Solaris maybe > FreeBSD) on Intel I7. > > And for the ring benchmark (single ring) is it not so that it sends > a message in a ring so there is only one process at every instant that > can execute. ?So all the SMP emulator can contribute is overhead, making > this benchmark only measure how much overhead the SMP emulator has > for pure message passing and process scheduling. The SMP emulator > can never beat the non-SMP emulator for a single ring and it can > only load one CPU no more than 100%. > > > > > smp 2:2 -> 18003 ms > > smp disabled -> 4867 ms > > 2 os processes -> ~5900 ms > > > hth, > > Dave > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From johanmon@REDACTED Fri May 7 10:56:16 2010 From: johanmon@REDACTED (Johan Montelius) Date: Fri, 07 May 2010 10:56:16 +0200 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: Message-ID: On Thu, 06 May 2010 23:52:57 +0200, Kenneth Lundin wrote: > On the other hand there are many examples of real applications that > really run well > in a single smp enabled VM on a multicore HW, Agree that this is the case and I would accept a small performance hit to have smp enabled as default but if the execution time can double or quadruple just because there is not enough or to fined concurrency then I think one should have smp disabled as default. Dual core is almost standard today and quadcore is soon in the laptops of my students. Many of them don't even think about multicore parallelism but still would compare runtimes with other systems or even try to analyze their run-times and see how they compare with their calculated figures from a model (which will not work since they don't take scheduling thrashing into account). How many even do experiments with smp disabled, their might be many examples of real application out there that could cut the execution time in two if they tried turning smp off. I'm sure this will improve and I'll be willing to help. Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From bengt.kleberg@REDACTED Fri May 7 11:49:27 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 07 May 2010 11:49:27 +0200 Subject: how: fread/2 with "-" in the input? Message-ID: <1273225767.4972.21.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, I have the following string: "2-1". I want to read the 2 numbers, like this (warning, this does not work); {ok, [Year, Month], _T} = io_lib:fread( "~i-~i", "2-1" ). I get a bad format. Presumably since "-" can be part of the control format? Is there a way to tell fread/2 that I want "-", not a control bengt From rumata-estor@REDACTED Fri May 7 11:58:53 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Fri, 07 May 2010 13:58:53 +0400 Subject: [erlang-questions] how: fread/2 with "-" in the input? In-Reply-To: <1273225767.4972.21.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> References: <1273225767.4972.21.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Message-ID: <4BE3E45D.7040905@nm.ru> {ok, [Year, Month], _T} = io_lib:fread( "~d-~d", "2-1" ). {ok,[2,1],[]} Bengt Kleberg wrote: > Greetings, > > I have the following string: "2-1". I want to read the 2 numbers, like > this (warning, this does not work); > {ok, [Year, Month], _T} = io_lib:fread( "~i-~i", "2-1" ). > > I get a bad format. Presumably since "-" can be part of the control > format? > > Is there a way to tell fread/2 that I want "-", not a control > > > bengt > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From michael.santos@REDACTED Fri May 7 13:14:25 2010 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 7 May 2010 07:14:25 -0400 Subject: [erlang-questions] Re: How to tell epmd to release a name rigsitered but not valid any more? In-Reply-To: References: <23f91031003032336v4d9bea58vc506210697af61ff@mail.gmail.com> Message-ID: <20100507111425.GA5429@ecn.lan> On Fri, May 07, 2010 at 10:31:32AM +0800, Seven Du wrote: > Was there anyone but me has this problem? I have met quite a few times. > > $ which erl > /usr/local/bin/erl > $ erl -v > Erlang R13B02 (erts-5.7.3) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.7.3 (abort with ^G) > > $ which epmd > /usr/local/bin/epmd > > $ epmd -names > epmd: up and running on port 4369 with data: > name freeswitch at port 8031 > name mos at port 61573 > name queue at port 58107 > name queue1 at port 16557 > name voiceEvents at port 57340 > > For some reason I stoped queue, How did you stop queue? > $ netstat -an|grep 4369 > tcp 0 0 127.0.0.1:4369 127.0.0.1:23526 ESTABLISHED > tcp 0 0 127.0.0.1:4369 127.0.0.1:57900 ESTABLISHED > tcp 0 0 127.0.0.1:4369 127.0.0.1:5976 ESTABLISHED > tcp 0 0 127.0.0.1:4369 127.0.0.1:30721 ESTABLISHED > tcp 0 0 127.0.0.1:4369 127.0.0.1:42265 ESTABLISHED > epmd: up and running on port 4369 with data: > name freeswitch at port 8031 > name mos at port 61573 > name queue at port 58107 > name queue1 at port 16557 > name voiceEvents at port 57340 There are 5 connections in established state to epmd and 5 registered names. Is the "queue" erlang node still running? You can check which processes are holding the sockets open using lsof (lsof -i tcp:4369) From dmitriid@REDACTED Fri May 7 13:43:12 2010 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Fri, 7 May 2010 14:43:12 +0300 Subject: A historical questions about binaries Message-ID: A friend of mine is digging deep into Erlang history and here's one question he hasn't found an answer to: At what time were binaries introduced in Erlang? I know that bit syntax appeared in 1999, it says so in the docs :) Were binaries introduced earlier? And what did they look like back then? From zabrane3@REDACTED Fri May 7 14:25:49 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Fri, 7 May 2010 14:25:49 +0200 Subject: Mnesia and IP addresses problem Message-ID: Hi, I'm facing a little issue whith my Mnesia DB. On one machine with IP addresse "ip1", i've created a persistent mnesia DB after running erlang in distributed mode "erl -name mynode1@REDACTED". When I copied that DB on another machine whith a different IP address (erl -node mynode@REDACTED), I was unable to read it. As you may notice, I've used the same node name "mynode" on both machines. Only IP are diffrent. What am I doing wrong? Regards Zabrane From rtrlists@REDACTED Fri May 7 14:39:26 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 7 May 2010 13:39:26 +0100 Subject: [erlang-questions] Mnesia and IP addresses problem In-Reply-To: References: Message-ID: On Fri, May 7, 2010 at 1:25 PM, zabrane Mikael wrote: > Hi, > > I'm facing a little issue whith my Mnesia DB. > On one machine with IP addresse "ip1", i've created a persistent mnesia DB > after > running erlang in distributed mode "erl -name mynode1@REDACTED". > > When I copied that DB on another machine whith a different IP address (erl > -node mynode@REDACTED), > I was unable to read it. > > As you may notice, I've used the same node name "mynode" on both machines. > Only IP > are diffrent. > > What am I doing wrong? > > Regards > Zabrane > Mnesia schemas are node specific. When you are migrating between machines (and you using long names or something else than @localhost) or change the node name you have to convert the schema to follow. See http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id2277864 for more information on how to do that. Obviously, you only need to do that if you have data that needs to get migrated. If you don't care about the data, just remove all the mnesia files and rebuild the schema from scratch. Robby From marcel.meyer@REDACTED Fri May 7 15:00:41 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Fri, 7 May 2010 09:00:41 -0400 Subject: Processes memory Message-ID: Hi there, I have been playing around with the infamous "perms" example where all permutations of a word is computed using this code: perms([]) -> [[]]; perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])]. Before I run a calculation, I get the memory dump for the shell: 2> memory(). [{total,4309984}, {processes,414604}, {processes_used,408188}, {system,3895380}, {atom,292069}, {atom_used,260714}, {binary,324936}, {code,1846671}, {ets,129732}] Then I run it: 3> test:perms("processese"). The issue is that I can only run this command once, after which a crash dump is produced, talking about memory allocation that fails. If I do a memory dump of the shell after executing the command once, this is what I get: 4> memory(). [{total,867872760}, {processes,864249884}, {processes_used,864243468}, {system,3622876}, {atom,298605}, {atom_used,268388}, {binary,24192}, {code,1867490}, {ets,130684}] The "processes*" have increased significantly. I did not bind the result of that computation, so why is it still consuming resources? A side note: I posted the perms problem to all our developers in the office (a .Net shop), and the fastest version (F#) was still twice as slow as the code above. Thank you in advance. Marcel From rvirding@REDACTED Fri May 7 15:27:31 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 7 May 2010 15:27:31 +0200 Subject: [erlang-questions] Processes memory In-Reply-To: References: Message-ID: On 7 May 2010 15:00, Marcel Meyer wrote: > Hi there, > I have been playing around with the infamous "perms" example where all > permutations of a word is computed using this code: > > perms([]) -> [[]]; > perms(L) -> [[H|T] || H <- L, T <- perms(L--[H])]. > > ... > The "processes*" have increased significantly. > I did not bind the result of that computation, so why is it still consuming > resources? The shell keeps a history list of past expressions and values so you can retrieve the values of previous computations. This even if you do not explicitly bind the result to a variable. To get around this try doing something after the perm call but in the same shell expression: 4> test:perm("processese"), 42. This should calculate the perms but not save them anywhere. You can also do: 5> Ps = test:perm("processese"), 42. which binds Ps to them but still does not put them on the history list. > A side note: I posted the perms problem to all our developers in the office > (a .Net shop), and the fastest version (F#) was still twice as slow as the > code above. That is very interesting and gratifying. Robert P.S. No Erlang here so I can't test this. From james.hague@REDACTED Fri May 7 15:29:10 2010 From: james.hague@REDACTED (James Hague) Date: Fri, 7 May 2010 08:29:10 -0500 Subject: [erlang-questions] Processes memory In-Reply-To: References: Message-ID: > The "processes*" have increased significantly. > I did not bind the result of that computation, so why is it still consuming > resources? The results of previous expressions are saved in the shell, so you can refer to them with the v(N) command, where N is the entry in the history list. You can change how many results are saved with resullts(N). Both of these commands are in the shell help: help(). From james.hague@REDACTED Fri May 7 15:30:42 2010 From: james.hague@REDACTED (James Hague) Date: Fri, 7 May 2010 08:30:42 -0500 Subject: [erlang-questions] Re: dividing a project across directories In-Reply-To: References: Message-ID: Thanks all! I didn't know about the "outdir" option. And I'm also inspired to switch over to the make module. James From marcel.meyer@REDACTED Fri May 7 15:49:52 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Fri, 7 May 2010 09:49:52 -0400 Subject: [erlang-questions] Processes memory In-Reply-To: References: Message-ID: Thank you for the quick response. @James, setting the shell to results(0) worked, thank you! @Robert, I tried the "test:perm("processese"), 42." trick but it did not work. I saw that only 42 was returned but the memory dump still showed allocation. Thanks for your help though! Regards, Marcel On 7 May 2010 09:29, James Hague wrote: > > The "processes*" have increased significantly. > > I did not bind the result of that computation, so why is it still > consuming > > resources? > > The results of previous expressions are saved in the shell, so you can > refer to them with the v(N) command, where N is the entry in the > history list. You can change how many results are saved with > resullts(N). Both of these commands are in the shell help: help(). > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rvirding@REDACTED Fri May 7 15:55:55 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 7 May 2010 15:55:55 +0200 Subject: [erlang-questions] Processes memory In-Reply-To: References: Message-ID: On 7 May 2010 15:49, Marcel Meyer wrote: > Thank you for the quick response. > > @James, setting the shell to results(0) worked, thank you! > @Robert, I tried the "test:perm("processese"), 42." trick but it did not > work. I saw that only 42 was returned but the memory dump still showed > allocation. Thanks for your help though! Yes, but did it still crash when running it again? Robert From marcel.meyer@REDACTED Fri May 7 16:06:17 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Fri, 7 May 2010 10:06:17 -0400 Subject: [erlang-questions] Processes memory In-Reply-To: References: Message-ID: Actually, the main question is that, if I had a hypothetical "perms" server, and I had to bind to the results for some reason, would the server run out of memory after 2 client requests? On 7 May 2010 09:49, Marcel Meyer wrote: > Thank you for the quick response. > > @James, setting the shell to results(0) worked, thank you! > @Robert, I tried the "test:perm("processese"), 42." trick but it did not > work. I saw that only 42 was returned but the memory dump still showed > allocation. Thanks for your help though! > > Regards, > Marcel > > On 7 May 2010 09:29, James Hague wrote: > >> > The "processes*" have increased significantly. >> > I did not bind the result of that computation, so why is it still >> consuming >> > resources? >> >> The results of previous expressions are saved in the shell, so you can >> refer to them with the v(N) command, where N is the entry in the >> history list. You can change how many results are saved with >> resullts(N). Both of these commands are in the shell help: help(). >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From dmercer@REDACTED Fri May 7 16:37:09 2010 From: dmercer@REDACTED (David Mercer) Date: Fri, 7 May 2010 09:37:09 -0500 Subject: [erlang-questions] Re: dividing a project across directories In-Reply-To: References: Message-ID: <9A4F8A004587413DB6C3C62D3B6FC71C@SSI.CORP> On Friday, May 07, 2010, James Hague wrote: > Thanks all! I didn't know about the "outdir" option. And I'm also > inspired to switch over to the make module. I've always used traditional make, but this discussion has inspired me to take a look, too. Anyone know of a good getting-started guide for Erlang's make? Thanks, David From tuncer.ayaz@REDACTED Fri May 7 16:51:57 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 7 May 2010 16:51:57 +0200 Subject: [erlang-questions] Re: dividing a project across directories In-Reply-To: References: Message-ID: On Fri, May 7, 2010 at 3:08 AM, Steve Davis wrote: > The following wildcards will work in Emakefiles: > > {"src/*", [{i, "include"}, {outdir, "ebin"}, debug_info, > strict_record_tests]}. > {"src/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, > strict_record_tests]}. > > I think that's what you're asking for! It's interesting that "src/*" etc. works in an Emakefile executed via 'erl -make' but specifying the same filespec fails if you directly call make:files/1,2. From tuncer.ayaz@REDACTED Fri May 7 16:53:25 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 7 May 2010 16:53:25 +0200 Subject: [erlang-questions] Re: dividing a project across directories In-Reply-To: References: Message-ID: On Fri, May 7, 2010 at 4:51 PM, Tuncer Ayaz wrote: > On Fri, May 7, 2010 at 3:08 AM, Steve Davis > wrote: >> The following wildcards will work in Emakefiles: >> >> {"src/*", [{i, "include"}, {outdir, "ebin"}, debug_info, >> strict_record_tests]}. >> {"src/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, >> strict_record_tests]}. >> >> I think that's what you're asking for! > > It's interesting that "src/*" etc. works in an Emakefile > executed via 'erl -make' but specifying the same filespec > fails if you directly call make:files/1,2. make:files("src/*", Opts) make:files("src/*/*", Opts) make:files('src/*', Opts) make:files('src/*/*', Opts) From comptekki@REDACTED Fri May 7 16:58:47 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 7 May 2010 08:58:47 -0600 Subject: [erlang-questions] strip end off of string In-Reply-To: <77A1754B-3894-4641-A0B8-9D5E0787D946@cs.otago.ac.nz> References: <77A1754B-3894-4641-A0B8-9D5E0787D946@cs.otago.ac.nz> Message-ID: On Thu, May 6, 2010 at 11:30 PM, Richard O'Keefe wrote: Richard, Thank you. I appreciate your on this:) -wes From loris.fichera@REDACTED Fri May 7 17:43:44 2010 From: loris.fichera@REDACTED (kid_a) Date: Fri, 07 May 2010 16:43:44 +0100 Subject: starting multiple erlang nodes with a single command In-Reply-To: References: Message-ID: <4BE43530.4030305@gmail.com> Hello there, think it it my first post on this mailing list. :) I'm working on rosen [1], a robotic simulation engine that makes use of openGL, sdl [2] and its erlang bindings [3] in order to provide a graphical representation of the simulated environment. What I'm trying to do is to add support for stereoscopic vision, providing rosen with the ability to display two windows, each displaying the environment from a certain point of view. Unfortunately, the latest stable version of sdl (1.2.14) does not support multi-windowed application. So, to avoid the single-window limit, I thought about the following trick: to move the "graphical engine" of rosen on two different erlang nodes. It seems to work. So far, I've been forced to manually launch the nodes to have the application up and running, while I'd like to launch the entire system with a single command - e.g. using a Makefile. Is there a way to do so? Any other ideas/advices/suggestions? :) Loris [1] http://sourceforge.net/projects/rosen/ [2] http://www.libsdl.org/ [3] http://esdl.sourceforge.net/ -- [ "let me take you down, 'cos i'm going to Strawberry Fields." ] [ Loris Fichera a.k.a. kid_a | HOME http://www.kida.netsons.org ] [ Visiting Student @ Uni of Herts | l.fichera@REDACTED ] [ JABBER loris[dot]fichera[at]gmail[dot]com |GPG key ID 179FC4DD ] From spkhaira@REDACTED Fri May 7 17:56:32 2010 From: spkhaira@REDACTED (Sant Parkash Singh) Date: Fri, 7 May 2010 21:26:32 +0530 Subject: [erlang-questions] Re: dividing a project across directories In-Reply-To: <9A4F8A004587413DB6C3C62D3B6FC71C@SSI.CORP> References: <9A4F8A004587413DB6C3C62D3B6FC71C@SSI.CORP> Message-ID: I guess this man page is quite good to start with - http://www.erlang.org/doc/man/make.html On Fri, May 7, 2010 at 8:07 PM, David Mercer wrote: > On Friday, May 07, 2010, James Hague wrote: > > > Thanks all! I didn't know about the "outdir" option. And I'm also > > inspired to switch over to the make module. > > I've always used traditional make, but this discussion has inspired me to > take a look, too. Anyone know of a good getting-started guide for Erlang's > make? > > Thanks, > > David > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jon@REDACTED Fri May 7 18:33:13 2010 From: jon@REDACTED (Jon Meredith) Date: Fri, 07 May 2010 10:33:13 -0600 Subject: [erlang-questions] starting multiple erlang nodes with a single command In-Reply-To: <4BE43530.4030305@gmail.com> References: <4BE43530.4030305@gmail.com> Message-ID: <4BE440C9.7080807@jonmeredith.com> Have you looked at the slave() module? If you're using X11 you might be able to do something like this from inside erlang start_displays() -> {ok, D1} = slave:start(net_adm:localhost(), display1, " -env DISPLAY " ++ net_adm:localhost() ++ ":1 "), {ok, D2} = slave:start(net_adm:localhost(), display2, " -env DISPLAY " ++ net_adm:localhost() ++ ":2 "), {D1, D2}. You'll need to start erlang as a named node (erl -name master), then you should be able to access them using the node names in the return tuple. Then just use rpc:call to start up whatever you need (or tweak the startup args if you prefer). Jon. On 5/7/10 9:43 AM, kid_a wrote: > Hello there, > > think it it my first post on this mailing list. :) > > I'm working on rosen [1], a robotic simulation engine that makes use of > openGL, sdl [2] and its erlang bindings [3] in order to provide a > graphical representation of the simulated environment. > What I'm trying to do is to add support for stereoscopic vision, > providing rosen with the ability to display two windows, each displaying > the environment from a certain point of view. > > Unfortunately, the latest stable version of sdl (1.2.14) does not > support multi-windowed application. So, to avoid the single-window > limit, I thought about the following trick: to move the "graphical > engine" of rosen on two different erlang nodes. It seems to work. > > So far, I've been forced to manually launch the nodes to have the > application up and running, while I'd like to launch the entire system > with a single command - e.g. using a Makefile. Is there a way to do so? > > Any other ideas/advices/suggestions? :) > > Loris > > [1] http://sourceforge.net/projects/rosen/ > [2] http://www.libsdl.org/ > [3] http://esdl.sourceforge.net/ > From tony.arcieri@REDACTED Fri May 7 19:41:27 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Fri, 7 May 2010 11:41:27 -0600 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: <20100507072748.GA14799@erix.ericsson.se> References: <20100507072748.GA14799@erix.ericsson.se> Message-ID: On Fri, May 7, 2010 at 1:27 AM, Raimo Niskanen < raimo+erlang-questions@REDACTED > wrote: > There are many different reasons why SMP, especially SMP benchmarks > (as Kenneth explained in another mail in this thread) performs poorly. > Wouldn't some kind of scheduler affinity for processes that send a lot of messages to each other and don't consume a lot of CPU be helpful in cases like this? -- Tony Arcieri Medioh! A Kudelski Brand From loris.fichera@REDACTED Fri May 7 20:28:26 2010 From: loris.fichera@REDACTED (kid_a) Date: Fri, 07 May 2010 19:28:26 +0100 Subject: [erlang-questions] starting multiple erlang nodes with a single command In-Reply-To: <4BE440C9.7080807@jonmeredith.com> References: <4BE43530.4030305@gmail.com> <4BE440C9.7080807@jonmeredith.com> Message-ID: <4BE45BCA.3030208@gmail.com> Jon Meredith wrote: > Have you looked at the slave() module? > > If you're using X11 you might be able to do something like this from > inside erlang > > start_displays() -> > {ok, D1} = slave:start(net_adm:localhost(), display1, " -env DISPLAY " > ++ net_adm:localhost() ++ ":1 "), > {ok, D2} = slave:start(net_adm:localhost(), display2, " -env DISPLAY " > ++ net_adm:localhost() ++ ":2 "), > {D1, D2}. > > You'll need to start erlang as a named node (erl -name master), then you > should be able to access them using the node names in the return tuple. > Then just use rpc:call to start up whatever you need (or tweak the > startup args if you prefer). It seems to be what I was looking for. Thank you. :) Loris -- [ "let me take you down, 'cos i'm going to Strawberry Fields." ] [ Loris Fichera a.k.a. kid_a | HOME http://www.kida.netsons.org ] [ Visiting Student @ Uni of Herts | l.fichera@REDACTED ] [ JABBER loris[dot]fichera[at]gmail[dot]com |GPG key ID 179FC4DD ] From comptekki@REDACTED Fri May 7 22:09:07 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 7 May 2010 14:09:07 -0600 Subject: mnesia and nul data Message-ID: I am trying to learn how to use mnesia. I have some test data from a csv file that I wrapped in {table_name, field1, field2, .... } and I'm using the company.erl example as an example to insert this data in to the test table. I've been looking around, but I don't see how to put nul data in to mnesia as compiler doesn't seem to like "field,,field2" where ,, is the nul field value. What is the recommend way to put blank data in to mnesia? "" or _ or []? I have for the record: -record( qlibc, { id, title, author_editor, date_of_publication, publisher, collection_type, key_words, notes, call_number, publication_source, holdings, physical_placement } ). The insert function: fill_tables() -> Qlibc = [{qlibc,0,"Reference document: needs assessment for the food and agricultural sciences",[],1984,"Joint Council on food and Agricultural Sciences","Documents",[],[],"DOCUMENT A485",[]} ], [mnesia:dirty_write(W) || W <- Qlibc], ok. but I get: 25> qlibc:fill_tables(). ** exception exit: {aborted,{bad_type,{qlibc,0, "Reference document: needs assessment for the food and agricultural sciences", [],1984, "Joint Council on food and Agricultural Sciences", "Documents",[],[],"DOCUMENT A485",[]}}} in function mnesia:abort/1 in call from qlibc:'-fill_tables/0-lc$^0/1-0-'/1 in call from qlibc:fill_tables/0 What's going on? thx, -wes From comptekki@REDACTED Fri May 7 22:16:57 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 7 May 2010 14:16:57 -0600 Subject: mnesia and nul data In-Reply-To: References: Message-ID: looks like I was missing a couple of columns of data. I'd still like to know the default nul or blank value to put in a field. thx, -wes From prikrutil@REDACTED Fri May 7 22:16:49 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sat, 8 May 2010 00:16:49 +0400 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: > I personally don't think that we need or should have a special syntax > for dicts After some thinking and rereading I should admit that Richard O'Keefe is right and what I actually need is more like frames/structs than dictionaries (actually, I don't even need indexes on keys, because my documents are quite small). Garbage collected atoms + frames/structs are enough. With atoms not being garbage collected frames/stuct can only be *almost* dynamic, because using them for representing deep and dynamic data structures (JSON/XML/YAML) will soon lead to leaking atoms. -- Sergey Samokhin From dmurray@REDACTED Fri May 7 22:11:18 2010 From: dmurray@REDACTED (David N Murray) Date: Fri, 07 May 2010 16:11:18 -0400 (EDT) Subject: TCP receive in gen_server Message-ID: Hi, I have a gen_server that opens a socket to another (non-erlang) process that it talks to over a TCP socket. I open the socket with case gen_tcp:connect(Host, Port, [list, inet]) of and I expect that I will see data from the socket showing up in my handle_info({tcp, Socket, Data}) function. I have another gen_server that performs the writes to the Socket. After the reader process connects, it passes a message to the writer with the Socket returned from the connect(). Everything seems to be working fine, especially reconnecting after a crash (the supervisor/gen_server combination is great!). However, after the initial handshake the only time I receive data via handle_info is after I write something to the socket a second time. This results in a delayed conversation: write1, receive informational messages that were sitting in socket (not reply to write1), write2, receive reply to write1, write3, receive reply to write2, etc. I know the reply is being immediately passed back from the server process I'm connecting; I can see it in that program's log. Am I missing something about how this should work? TIA, Dave From nem@REDACTED Fri May 7 22:31:24 2010 From: nem@REDACTED (Geoff Cant) Date: Fri, 7 May 2010 13:31:24 -0700 Subject: [erlang-questions] Re: mnesia and nul data In-Reply-To: References: Message-ID: <650585D8-D10B-4222-9B55-7863FD4154D4@erlang.geek.nz> There is no null value in erlang that has any special semantic meaning[1] - by convention we tend to use the atom 'undefined' but you could use anything you like. If your record definitions don't specify default values '-record(some_rec, {foo = default, bar = 12, baz}', then they'll default to 'undefined' (as #some_rec.baz does in that example). In fact, I would suggest that you use some descriptive atom that notes why there is no value; using 'not_applicable', 'pending', 'waived', 'unknown' has no cost over using 'undefined' and may make debugging data issues easier later. Cheers, -Geoff [1] neither erlang nor mnesia implements the awful three-value-logic that SQL uses (true/false/null). On 2010-05-07, at 13:16 , Wes James wrote: > looks like I was missing a couple of columns of data. I'd still like > to know the default nul or blank value to put in a field. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From kagato@REDACTED Fri May 7 22:33:00 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Fri, 7 May 2010 13:33:00 -0700 Subject: [erlang-questions] Special syntax for dictionaries In-Reply-To: References: <66490FD3-4B40-472B-BE8B-87AFFCC45447@cs.otago.ac.nz> Message-ID: <02DDF127-28B6-449C-BB94-03DD2B420D89@souja.net> Recently, I submitted a patch to make term_to_binary have a "safe" option (i.e. doesn't create new atoms). It was accepted. My use case was decoding binaries from the network and dropping them into a pattern-match to make simple network protocols. It turned out that I had no problem just creating the atoms I needed and letting the conversion fail if I got a bad atom. Perhaps you could do something similar in the JSON or YAML decoder? Just use list_to_existing_atom/1. If it fails, collect that key into a 'unknown keys' dictionary or just throw it away if you don't need to save / use the extra data. It would be perhaps a little ugly, but it should work. I've always found that a little judicious design can generally prevent atom leakage. Good luck! -- Jayson Vantuyl On May 7, 2010, at 1:16 PM, Sergey Samokhin wrote: >> I personally don't think that we need or should have a special syntax >> for dicts > > After some thinking and rereading I should admit that Richard O'Keefe > is right and what I actually need is more like frames/structs than > dictionaries (actually, I don't even need indexes on keys, because my > documents are quite small). Garbage collected atoms + frames/structs > are enough. > > With atoms not being garbage collected frames/stuct can only be > *almost* dynamic, because using them for representing deep and dynamic > data structures (JSON/XML/YAML) will soon lead to leaking atoms. > > -- > Sergey Samokhin > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From comptekki@REDACTED Fri May 7 22:43:31 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 7 May 2010 14:43:31 -0600 Subject: [erlang-questions] Re: mnesia and nul data In-Reply-To: <650585D8-D10B-4222-9B55-7863FD4154D4@erlang.geek.nz> References: <650585D8-D10B-4222-9B55-7863FD4154D4@erlang.geek.nz> Message-ID: On Fri, May 7, 2010 at 2:31 PM, Geoff Cant wrote: > There is no null value in erlang that has any special semantic meaning[1] - by convention we tend to use the atom 'undefined' but you could use anything you like. If your record definitions don't specify default values '-record(some_rec, {foo = default, bar = 12, baz}', then they'll default to 'undefined' (as #some_rec.baz does in that example). > > In fact, I would suggest that you use some descriptive atom that notes why there is no value; using 'not_applicable', 'pending', 'waived', 'unknown' has no cost over using 'undefined' and may make debugging data issues easier later. > > Cheers, > -Geoff > [1] neither erlang nor mnesia implements the awful three-value-logic that SQL uses (true/false/null). OK thx. -wes From kaiduanx@REDACTED Sat May 8 01:18:09 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 7 May 2010 19:18:09 -0400 Subject: crypto:dss_sign() fails with bad argument Message-ID: Hi, I am playing with erlang's crypto module for DSS sign/verification, and encountered the following failure, kaiduanx@REDACTED:~/test$ erl Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.6.5 (abort with ^G) 1> {ok, [Key1]} = public_key:pem_to_der("./privkey.pem"). {ok,[{dsa_private_key,<<48,130,3,61,2,1,0,2,130,1,1,0,130, 155,217,13,154,35,30,61,35,193, 177,89,...>>, not_encrypted}]} 2> {ok, DSAPrivKey} = public_key:decode_private_key(Key1). {ok,{'DSAPrivateKey',0, 16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, 1069445346270440656593824092073657983328499138457, 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, 7633100674515027235650509950891694304003982881440195652180376335259908594946224965667327027587338158513959931360517774019595393102978917235479651037021972646784210191530035858681332371458695842841098509902898010434270664893194727388258476377361344595538044057461379480170807524999310663741254536687833627343418977980237734999549443057990979136155378902597245536582084684912096597468826612905623557372778470587564770314993903695513057146844763692020381754294541184291650680501093631643506634398260717309523197216269415879496293970907691201898118313013181084633883085546050181063797462116224600567526013743839898981226, 495667775549172824486294937719319801999514557700}} 3> {_, 0, P, Q, G, Y, X} = DSAPrivKey. {'DSAPrivateKey',0, 16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, 1069445346270440656593824092073657983328499138457, 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, 7633100674515027235650509950891694304003982881440195652180376335259908594946224965667327027587338158513959931360517774019595393102978917235479651037021972646784210191530035858681332371458695842841098509902898010434270664893194727388258476377361344595538044057461379480170807524999310663741254536687833627343418977980237734999549443057990979136155378902597245536582084684912096597468826612905623557372778470587564770314993903695513057146844763692020381754294541184291650680501093631643506634398260717309523197216269415879496293970907691201898118313013181084633883085546050181063797462116224600567526013743839898981226, 495667775549172824486294937719319801999514557700} 4> 4> crypto:start(). ok 5> crypto:dss_sign(list_to_binary("Welcome to erlang land"), [crypto:mpint(P), crypto:mpint(Q), crypto:mpint(G), crypto:mpint(X)]). ** exception error: bad argument in function port_control/3 called as port_control(crypto_drv02,38, [<<"Welcome to erlang land">>, [<<0,0,1,1,0,130,155,217,13,154,35,30,61,35, 193,177,89,67,155,111,87,191,198,...>>, <<0,0,0,21,0,187,83,150,127,162,157,203,222, 11,119,150,240,120,83,23,248,64,...>>, <<0,0,1,0,57,38,127,21,93,84,216,239,141,178, 131,95,23,15,1,197,166,...>>, <<0,0,0,20,86,210,131,3,27,84,85,4,140,24, 92,36,218,245,126,21,...>>]]) in call from crypto:control/2 in call from crypto:dss_sign/2 6> crypto:dss_sign(list_to_binary("Welcome to erlang land"), [P, Q, G, X]). ** exception error: bad argument in function port_control/3 called as port_control(crypto_drv02,38, [<<"Welcome to erlang land">>, [16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, 1069445346270440656593824092073657983328499138457, 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, 495667775549172824486294937719319801999514557700]]) in call from crypto:control/2 in call from crypto:dss_sign/2 7> 9> crypto:info_lib(). [{<<"OpenSSL">>,9470079,<<"OpenSSL 0.9.8g 19 Oct 2007">>}] The privkey.pem is generated by openssl as below, openssl dsaparam -out dsaparam.pem 2048 openssl gendsa -out privkey.pem dsaparam.pem Can someone point out what is going wrong? Thanks for help, Kaiduan From kaiduanx@REDACTED Sat May 8 04:12:26 2010 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Fri, 7 May 2010 22:12:26 -0400 Subject: crypto:dss_sign() fails with bad argument In-Reply-To: References: Message-ID: After looking the file pubkey_crypto.erl in Erlang/OTP source, I found the right way as below, crypto:dss_sign(sized_binary(Bin), [P, Q, G, X]). sized_binary(Bin) when is_binary(Bin) -> <<(size(Bin)):32/integer, Bin/binary>>; sized_binary(List) when is_list(List) -> sized_binary(list_to_binary(List)). Another question comes naturally, given a public DSA key in PEM format to verify, how to get the [P, Q, G, Y] from the public key? 1> public_key:pem_to_der("./pub.pem"). {ok,[]} 2> public_key:pem_to_der("./dsaparam.pem"). {ok,[]} Thanks, Kaiduan On Fri, May 7, 2010 at 7:18 PM, Kaiduan Xie wrote: > Hi, > > I am playing with erlang's crypto module for DSS sign/verification, > and encountered the following failure, > > kaiduanx@REDACTED:~/test$ erl > Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.6.5 ?(abort with ^G) > 1> {ok, [Key1]} = public_key:pem_to_der("./privkey.pem"). > {ok,[{dsa_private_key,<<48,130,3,61,2,1,0,2,130,1,1,0,130, > ? ? ? ? ? ? ? ? ? ? ? ?155,217,13,154,35,30,61,35,193, > ? ? ? ? ? ? ? ? ? ? ? ?177,89,...>>, > ? ? ? ? ? ? ? ? ? ? ?not_encrypted}]} > 2> {ok, DSAPrivKey} = public_key:decode_private_key(Key1). > {ok,{'DSAPrivateKey',0, > > 16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, > ? ? ? ? ? ? ? ? ? ? 1069445346270440656593824092073657983328499138457, > > 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, > > 7633100674515027235650509950891694304003982881440195652180376335259908594946224965667327027587338158513959931360517774019595393102978917235479651037021972646784210191530035858681332371458695842841098509902898010434270664893194727388258476377361344595538044057461379480170807524999310663741254536687833627343418977980237734999549443057990979136155378902597245536582084684912096597468826612905623557372778470587564770314993903695513057146844763692020381754294541184291650680501093631643506634398260717309523197216269415879496293970907691201898118313013181084633883085546050181063797462116224600567526013743839898981226, > ? ? ? ? ? ? ? ? ? ? 495667775549172824486294937719319801999514557700}} > 3> {_, 0, P, Q, G, Y, X} = DSAPrivKey. > {'DSAPrivateKey',0, > > 16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, > ? ? ? ? ? ? ? ? 1069445346270440656593824092073657983328499138457, > > 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, > > 7633100674515027235650509950891694304003982881440195652180376335259908594946224965667327027587338158513959931360517774019595393102978917235479651037021972646784210191530035858681332371458695842841098509902898010434270664893194727388258476377361344595538044057461379480170807524999310663741254536687833627343418977980237734999549443057990979136155378902597245536582084684912096597468826612905623557372778470587564770314993903695513057146844763692020381754294541184291650680501093631643506634398260717309523197216269415879496293970907691201898118313013181084633883085546050181063797462116224600567526013743839898981226, > ? ? ? ? ? ? ? ? 495667775549172824486294937719319801999514557700} > 4> > 4> crypto:start(). > ok > 5> crypto:dss_sign(list_to_binary("Welcome to erlang land"), > [crypto:mpint(P), crypto:mpint(Q), crypto:mpint(G), crypto:mpint(X)]). > ** exception error: bad argument > ? ? in function ?port_control/3 > ? ? ? ?called as port_control(crypto_drv02,38, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [<<"Welcome to erlang land">>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[<<0,0,1,1,0,130,155,217,13,154,35,30,61,35, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 193,177,89,67,155,111,87,191,198,...>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<0,0,0,21,0,187,83,150,127,162,157,203,222, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 11,119,150,240,120,83,23,248,64,...>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<0,0,1,0,57,38,127,21,93,84,216,239,141,178, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 131,95,23,15,1,197,166,...>>, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? <<0,0,0,20,86,210,131,3,27,84,85,4,140,24, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 92,36,218,245,126,21,...>>]]) > ? ? in call from crypto:control/2 > ? ? in call from crypto:dss_sign/2 > 6> crypto:dss_sign(list_to_binary("Welcome to erlang land"), [P, Q, G, > X]). ? ? ** exception error: bad argument > ? ? in function ?port_control/3 > ? ? ? ?called as port_control(crypto_drv02,38, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [<<"Welcome to erlang land">>, > > [16487831091520780258476244829269954905068453299919937468122754367555025389255228167104813916584427117132491697333745898571447582288126735030379964082170022817274086858448078061001221776403199146387597678984540700662102497749323871708629057309007645527617512892347216241068519054813731565592382595863130859729188869666097096597640296820469239977991443749290028015703576616678053020176398097226185345390078018458880065671569994747328759201284973167049924181380228215122295761433727402740292339377060448621018652887515687347119246401586369278290721630262294672892581614282203303348001277520129700110248972612083036183823, > > 1069445346270440656593824092073657983328499138457, > > 7214566675159847510543550105259613160453301579561623377843203423614163319395145087833198636004278609701319277802567885081948001327800441061451495445964179176778711411229815801134094069049261847343045298683209010702015484647378262734006939008757875094544682692811028223303902447231625138361611541322285116859770078463180719920891946211261933346941519603064902060705607105259990741513856339667300139111109012511141655066131159463312192937454414940901992301247028509228326669147276129260865537192028595304014436739105124361395299086087865530512935899515885409929656072537826761802885019421821204505764703508239164324041, > > 495667775549172824486294937719319801999514557700]]) > ? ? in call from crypto:control/2 > ? ? in call from crypto:dss_sign/2 > 7> > > 9> crypto:info_lib(). > [{<<"OpenSSL">>,9470079,<<"OpenSSL 0.9.8g 19 Oct 2007">>}] > > The privkey.pem is generated by openssl as below, > openssl dsaparam -out dsaparam.pem 2048 > openssl gendsa -out privkey.pem dsaparam.pem > > Can someone point out what is going wrong? Thanks for help, > > Kaiduan > From hd2010@REDACTED Sat May 8 09:43:00 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 08 May 2010 09:43:00 +0200 Subject: Cost of Copy Message-ID: <4BE51604.8020907@eonblast.com> 1 I am trying to get a better picture of what the performance costs are when mutating (collections of) complex structures. Most generally, if I have a list of structures and want to make one change to one of the structures; and if I want to avoid ETS for the implicit copying, what else should I NOT do to not run into the same problem with a non-ETS approach, and how CAN I have as little unnecessary internal copying of all those things that stay the same. I just ran into this post by J Louis while looking for information http://jlouisramblings.blogspot.com/2009/01/common-erlang-misconceptions.html It was giving some implicit answers about what results in internal copying, what doesn't. And I am not talking about strings here. I wondered if there is a dedicated write up out there that explains a little more to the point what the use of lists, sets, trees, ets and mnesia implies in terms of copying, to better understand the performance implications. With immutable variables, of course there is always the hope that underneath, not much copying is actually necessary, as I understand e.g. lists are benefiting hugely from, to become O(1) when adding heads to them. So assumptions based on ALGOL heritage experiences should fail. Which puts one on a quest like Max Lapshin's http://erlang.2086793.n4.nabble.com/Help-with-storing-data-in-memory-td2119208.html#a2119208 who is also really looking for a way to avoid implicit copying. I have been looking for clues in this direction all the time but found them rather scarce in the docs. May be me. May be an idea to put more focus on in erldocs, erlang.org etc? I am sure there are links. Maybe they could be featured more prominently. I am sure there are many people coming to Erlang with a deeply ingrained eye on performance and would like to understand these implications earlier in the learning experience. I mean things exactly like "changing status from x to y would copy the record, BUT the actual record structure is only a set of pointers." by Ulf Wiger in his post http://www.erlang.org/cgi-bin/ezmlm-cgi/4/1342 (var names changed) 2 From that post I understand that the best performance should come from a data structure that is as nested as possible, because that will reduce the amount of pointers that are copied with every mutation?! As the mutation copies not-changed data only on one level. That's a very interesting implication that so far I haven't found spelled out anywhere (but would have liked to learn up front, if true). I am still not sure if it would hold across the board. Or is it wrong for other reasons I am not aware of? (Or could this post maybe for any strange reason be ... outdated? It's a decade old in June. Wow!) 3 Surveying the choices, does this imply I am to use a copy-heavy approach if I want to have meaningful transaction support between processes? Because the choice there is ets, dets or mnesia and they all copy the entire structure everytime there is any access to it? 4 I would encourage a view that these kind of questions would be best discussed quite early in the learning curve. Together with the implications that immutable state has for message passing and how it saves so much copying in that case. Or so I speculate. I am not speaking about the error prevention aspects here. That said, thanks for all the efforts that are going into making the Erlang docs ever better! Even Google today adopted the classic format of the Erlang doc frame layout I realize. But seriously, thanks! Henning From prikrutil@REDACTED Sat May 8 13:37:56 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Sat, 8 May 2010 15:37:56 +0400 Subject: Special syntax for dictionaries In-Reply-To: References: Message-ID: Let's just *suppose* that frames/structs are already here. The question: are there any reasons why we would want to use records? In other words, should we now think of records as some kind of deprecated trick? To answer this question we should list features that only records have and then ask another question: "it this good to still have these features?". 1. With records you are restricted to declared keys. You can't add new keys on the fly. Are there situations when it's good? 2. With records you can specify default values to keys With frames it can'be achieved by using custom wrappers: create_my_frame(...) % Creates a new frame with all the default values specified -- Sergey Samokhin From rvirding@REDACTED Sat May 8 16:46:44 2010 From: rvirding@REDACTED (Robert Virding) Date: Sat, 8 May 2010 16:46:44 +0200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: Message-ID: I think there is really only one reason why you would use records instead of frames/structs: speed. Records will be faster as all the work needed to find out which field to access is done at compile-time. A secondary reason could be that you get some compile-time checking which will catch typos. I would expect that dialyzer would be able to catch some of those. I personally would convert and use frames/structs when they arrive. Robert On 8 May 2010 13:37, Sergey Samokhin wrote: > Let's just *suppose* that frames/structs are already here. > > The question: are there any reasons why we would want to use records? > In other words, should we now think of records as some kind of > deprecated trick? > > To answer this question we should list features that only records have > and then ask another question: "it this good to still have these > features?". > > 1. With records you are restricted to declared keys. You can't add new > keys on the fly. > > Are there situations when it's good? > > 2. With records you can specify default values to keys > > With frames it can'be achieved by using custom wrappers: > > create_my_frame(...) % Creates a new frame with all the default values specified > > -- > Sergey Samokhin > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Sat May 8 16:54:20 2010 From: comptekki@REDACTED (Wes James) Date: Sat, 8 May 2010 08:54:20 -0600 Subject: mnesia queries Message-ID: I have been looking at mnesia and can do some simple quieries. Are there queries like in sql where you have: where field like '%data' - that is a "like" with the "%" operator? thx, -wes From g@REDACTED Sat May 8 19:21:17 2010 From: g@REDACTED (Garrett Smith) Date: Sat, 8 May 2010 12:21:17 -0500 Subject: ordered_set and range queries Message-ID: I'm playing around with the performance characteristics of using type=ordered_set in mnesia. My record looks like #test{time, val} where time is an integer timestamp (e.g. epoch milliseconds). When I run this: mnesia:dirty_select(test, [{{test, '$1', '$2'}, [{'>=', '$1', Start}, {'=<', '$1', End}], [['$1', '$2']]}]] I get the same basic timings regardless of the result set size. I.e. I suspect there's a full table scan in every case. Just to verify, I ran the same tests on a type=set table and got the same basic timings. Is there a way to perform that select that takes advantage of the sorted key? Garrett From marcel.meyer@REDACTED Sat May 8 19:59:52 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Sat, 8 May 2010 13:59:52 -0400 Subject: [erlang-questions] mnesia queries In-Reply-To: References: Message-ID: Wes, I'm still digging into Erlang as well, but I would guess your predicate function will have to do a regex on the value. I hope the pros can suggest something better, as I read that Erlang's Regex lib is awfully slow. Regards, Marcel On 8 May 2010 10:54, Wes James wrote: > I have been looking at mnesia and can do some simple quieries. Are > there queries like in sql where you have: where field like '%data' - > that is a "like" with the "%" operator? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From marcel.meyer@REDACTED Sat May 8 20:27:24 2010 From: marcel.meyer@REDACTED (Marcel Meyer) Date: Sat, 8 May 2010 14:27:24 -0400 Subject: Erlang, Mnesia and Saas data isolation Message-ID: Hi there, I'm looking into Erlang and of course Mnesia to see whether and how it can be used to build a Saas platform. The interfacing bit seems generally easy: REST using YAWS looks feasible (thanks to some great talks from Steve Venoski). The data aspect pieces of this puzzle still seems opaque. One of the requirements is complete data segregation between clients, eg. 1 database per client. More than that, each client can have many data silos, which should also be kept separate from each other. The scaling trend is that both client databases and the client's silos will keep increasing. How would this look in Mnesia? How does # of databases scale using Mnesia? Can I load and start 10000's of Mnesia databases on a server? I know that Mnesia is also not the place to store ever growing data, so audit trails etc. might be moved out using the disk_log module. Any input will be greatly appreciated! Kind regards, Marcel From klacke@REDACTED Sat May 8 23:04:06 2010 From: klacke@REDACTED (Claes Wikstrom) Date: Sat, 08 May 2010 23:04:06 +0200 Subject: [erlang-questions] A historical questions about binaries In-Reply-To: References: Message-ID: <4BE5D1C6.7020303@hyber.org> On 05/07/2010 01:43 PM, Dmitrii Dimandt wrote: > A friend of mine is digging deep into Erlang history and here's one question he hasn't found an answer to: > > At what time were binaries introduced in Erlang? I know that bit syntax appeared in 1999, it says so in the docs :) Were binaries introduced earlier? And what did they look like back then? Binaries came a bit earlier, I introduced them during an effort to separate the emulator proper from OS calls and then in particular the file system calls that were used to load code. Thus came drivers, the whole driver arch, and in particular the file driver that replied with binaries that could be fed to the code loading BIFs. My guess would be 1995-1996 /klacke From hd2010@REDACTED Sat May 8 23:15:03 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 08 May 2010 23:15:03 +0200 Subject: [erlang-questions] A historical questions about binaries In-Reply-To: <4BE5D1C6.7020303@hyber.org> References: <4BE5D1C6.7020303@hyber.org> Message-ID: <4BE5D457.9080605@eonblast.com> This is scary. Erlang is not quite ancient just yet and the mysteries about its beginnings have already started to fade from human memory. Claes Wikstrom wrote: > On 05/07/2010 01:43 PM, Dmitrii Dimandt wrote: >> A friend of mine is digging deep into Erlang history and here's one >> question he hasn't found an answer to: >> >> At what time were binaries introduced in Erlang? I know that bit >> syntax appeared in 1999, it says so in the docs :) Were binaries >> introduced earlier? And what did they look like back then? > > Binaries came a bit earlier, I introduced them during an effort to > separate the emulator proper > from OS calls and then in particular the file system calls that were > used to load code. > Thus came drivers, the whole driver arch, and in particular the file > driver that replied with binaries > that could be fed to the code loading BIFs. > > My guess would be 1995-1996 > > /klacke > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hd2010@REDACTED Sun May 9 00:00:10 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 09 May 2010 00:00:10 +0200 Subject: Inline If Message-ID: <4BE5DEEA.5070208@eonblast.com> Is there a common Erlang way to do stuff like C++: a ? b : c; The list comprehension [X || X <- [B,C], A ] ? Thanks, Henning From ulf.wiger@REDACTED Sun May 9 00:04:42 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 09 May 2010 00:04:42 +0200 Subject: [erlang-questions] ordered_set and range queries In-Reply-To: References: Message-ID: <4BE5DFFA.60208@erlang-solutions.com> In order to limit the scan, you need to make sure the first part of the key pattern is bound/constant. In this case, only the record tag is constant, but it is the same for all records in the table... There is no analysis in place that determines whether the guard pattern could warrant a limited scan. For the kind of test you're doing here, an alternative would be to use mnesia:dirty_next(test, {test,Start,Floor}), where Floor is something that you know to be smaller than any actual value in that position in the table. Then you could keep doing next() until you hit the upper bound. Unless the table is really very large, this is likely to be more costly than doing the select, though. BR, Ulf W On 05/08/2010 07:21 PM, Garrett Smith wrote: > I'm playing around with the performance characteristics of using > type=ordered_set in mnesia. > > My record looks like #test{time, val} where time is an integer > timestamp (e.g. epoch milliseconds). > > When I run this: > > mnesia:dirty_select(test, [{{test, '$1', '$2'}, [{'>=', '$1', Start}, > {'=<', '$1', End}], [['$1', '$2']]}]] > > I get the same basic timings regardless of the result set size. I.e. I > suspect there's a full table scan in every case. > > Just to verify, I ran the same tests on a type=set table and got the > same basic timings. > > Is there a way to perform that select that takes advantage of the sorted key? > > Garrett > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From kiszl@REDACTED Sun May 9 00:25:16 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 09 May 2010 00:25:16 +0200 Subject: [erlang-questions] Inline If In-Reply-To: <4BE5DEEA.5070208@eonblast.com> References: <4BE5DEEA.5070208@eonblast.com> Message-ID: <4BE5E4CC.6030600@tmit.bme.hu> On 5/9/2010 12:00 AM, Henning Diedrich wrote: > Is there a common Erlang way to do stuff like C++: > > a ? b : c; > > The list comprehension > > [X || X <- [B,C], A ] > > ? > > Thanks, > Henning > You can define a function or macro for this purpose; see "assert" discussed in this thread: Ternary operator used as assert http://www.erlang.org/cgi-bin/ezmlm-cgi?4:sss:48575:200912:jchpnnbpampjdbbfpigm Regards, Zoltan. From hd2010@REDACTED Sun May 9 00:37:54 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 09 May 2010 00:37:54 +0200 Subject: [erlang-questions] Inline If In-Reply-To: <4BE5E4CC.6030600@tmit.bme.hu> References: <4BE5DEEA.5070208@eonblast.com> <4BE5E4CC.6030600@tmit.bme.hu> Message-ID: <4BE5E7C2.10005@eonblast.com> Thanks Zoltan, so the answer would be -define(ASSERT(TEST,TRUE,FALSE), case TEST of true -> TRUE; false -> FALSE end). This seems to imply that even just wanting to use such construct is not a common thing in Erlang. Is there a reason for that in the nature of FP or Erlang? Thanks, Henning Zoltan Lajos Kis wrote: > On 5/9/2010 12:00 AM, Henning Diedrich wrote: >> Is there a common Erlang way to do stuff like C++: >> >> a ? b : c; >> >> The list comprehension >> >> [X || X <- [B,C], A ] >> >> ? >> >> Thanks, >> Henning >> > > You can define a function or macro for this purpose; see "assert" > discussed in this thread: Ternary operator used as assert > http://www.erlang.org/cgi-bin/ezmlm-cgi?4:sss:48575:200912:jchpnnbpampjdbbfpigm > > > Regards, > Zoltan. > From kiszl@REDACTED Sun May 9 01:20:19 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 09 May 2010 01:20:19 +0200 Subject: [erlang-questions] Inline If In-Reply-To: <4BE5E7C2.10005@eonblast.com> References: <4BE5DEEA.5070208@eonblast.com> <4BE5E4CC.6030600@tmit.bme.hu> <4BE5E7C2.10005@eonblast.com> Message-ID: <4BE5F1B3.3090708@tmit.bme.hu> I guess the ternary operator only makes sense in non-functional languages, where conditional statements do not have return values. Given that there was no ternary operator, you would have to duplicate the assignment part, rendering the code less readable: if (a) { x = b; } else { x = c; }; /* x = a ? b : c; */ In a functional language, such as Erlang, the these statements do have a return value, so the ternary-operator would only be a syntactic sugar: X = case A of true -> B; false -> C end. Regarding your question, the answer depends on whether you need "lazy evaluation", and whether you like working with macros, or not. Regards, Zoltan. On 5/9/2010 12:37 AM, Henning Diedrich wrote: > Thanks Zoltan, > > so the answer would be > > -define(ASSERT(TEST,TRUE,FALSE), case TEST of true -> TRUE; false -> > FALSE end). > > This seems to imply that even just wanting to use such construct is > not a common thing in Erlang. > > Is there a reason for that in the nature of FP or Erlang? > > Thanks, > Henning > > > Zoltan Lajos Kis wrote: >> On 5/9/2010 12:00 AM, Henning Diedrich wrote: >>> Is there a common Erlang way to do stuff like C++: >>> >>> a ? b : c; >>> >>> The list comprehension >>> >>> [X || X <- [B,C], A ] >>> >>> ? >>> >>> Thanks, >>> Henning >>> >> >> You can define a function or macro for this purpose; see "assert" >> discussed in this thread: Ternary operator used as assert >> http://www.erlang.org/cgi-bin/ezmlm-cgi?4:sss:48575:200912:jchpnnbpampjdbbfpigm >> >> >> Regards, >> Zoltan. >> > From aj@REDACTED Sun May 9 01:38:33 2010 From: aj@REDACTED (AJ Heller) Date: Sat, 8 May 2010 16:38:33 -0700 Subject: beginning erlang/OTP - spawning gen_servers and supervisors Message-ID: I am reading through the OTP documentation, stopping to work with and understand every piece as it is introduced. I've written a very simple gen_server and a simpler supervisor, and they both work as I expect them to, but I've had to stop there. I'd like to understand them each before I move on to building an application, but I'm confused about the behavior of the system when I spawn each of these pieces independently. (For reference, my gen_server (echo_server) and supervisor (echo_sup) modules are at the end of this post) I am able to spawn my simple gen_server from my erl shell using `spawn(echo_server, start_link, []).` I can then interact with it using its public api (`echo_server:echo(...)`). However, I cannot spawn a supervisor the same way. By calling `spawn(echo_sup, start_link, []).`, my supervisor and gen_server both appear to exit immediately after they have finished initializing. Everything looks fine until I try to interact with my gen_server and get "** exception exit: {noproc,{gen_server,call,[echo_server,{echo,hi}]}} in function gen_server:call/2" Why is it that I can spawn a gen_server but not a supervisor? Thank you much. --------------------------------------------------------------------------------------------------------- echo_server: % -- begin echo_server.erl -- % -module(echo_server). -behaviour(gen_server). -export([start_link/0]). -export([echo/1, crash/0]). -export([init/1, handle_call/3, handle_cast/2]). start_link() -> gen_server:start_link({local, echo_server}, echo_server, [], []). %% public api echo(Text) -> gen_server:call(echo_server, {echo, Text}). %% crash this gen_server to test its supervisor crash() -> gen_server:call(echo_server, crash). %% behaviours init(_Args) -> {ok, none}. handle_call({echo, Text}, _From, State) -> {reply, Text, State}; handle_call(crash, _From, State) -> X=1, {reply, X=2, State}. handle_cast(_, State) -> {noreply, State}. % -- end echo_server.erl -- % echo_sup: % -- begin echo_sup.erl -- % -module(echo_sup). -behaviour(supervisor). -export([start_link/0]). -export([init/1]). start_link() -> supervisor:start_link({local, echo_sup}, echo_sup, []). init(_Args) -> {ok, {{one_for_one, 5, 60}, [{echo_server, {echo_server, start_link, []}, permanent, brutal_kill, worker, [echo_server]}]}}. % -- end echo_sup.erl -- % From nem@REDACTED Sun May 9 02:19:42 2010 From: nem@REDACTED (Geoff Cant) Date: Sat, 8 May 2010 17:19:42 -0700 Subject: [erlang-questions] beginning erlang/OTP - spawning gen_servers and supervisors In-Reply-To: References: Message-ID: <8F6C0C8C-FBB8-4403-B9DD-24A9D7CBEBE6@erlang.geek.nz> If you do spawn(echo_sup, start_link, []), you'll spawn a process (A) that will spawn and link to an echo_sup supervisor process (B) which will spawn your gen_server (C). A will then exit because it has no more code to run (ala exit(normal)), causing B to exit (due to the link to A), causing C to exit (due to the link to B). You should either run echo_sup:start_link() at the shell (and then unlink the pid returned as otherwise mistakes you type in the shell will kill echo_sup) or you want to start echo_sup underneath an already running supervisor. (Or you want to write an application callback echo_app, and an echo.app file i.e. build an echo_server OTP application and have the application_controller deal with the top level of supervision). Cheers, -Geoff On 2010-05-08, at 16:38 , AJ Heller wrote: > I am reading through the OTP documentation, stopping to work with and > understand every piece as it is introduced. I've written a very simple > gen_server and a simpler supervisor, and they both work as I expect > them to, but I've had to stop there. I'd like to understand them each > before I move on to building an application, but I'm confused about > the behavior of the system when I spawn each of these pieces > independently. > > (For reference, my gen_server (echo_server) and supervisor (echo_sup) > modules are at the end of this post) > > I am able to spawn my simple gen_server from my erl shell using > `spawn(echo_server, start_link, []).` I can then interact with it > using its public api (`echo_server:echo(...)`). > > However, I cannot spawn a supervisor the same way. By calling > `spawn(echo_sup, start_link, []).`, my supervisor and gen_server both > appear to exit immediately after they have finished initializing. > Everything looks fine until I try to interact with my gen_server and > get > > "** exception exit: {noproc,{gen_server,call,[echo_server,{echo,hi}]}} > in function gen_server:call/2" > > Why is it that I can spawn a gen_server but not a supervisor? Thank you much. > > > --------------------------------------------------------------------------------------------------------- > echo_server: > > % -- begin echo_server.erl -- % > > -module(echo_server). > -behaviour(gen_server). > -export([start_link/0]). > -export([echo/1, crash/0]). > -export([init/1, handle_call/3, handle_cast/2]). > > start_link() -> > gen_server:start_link({local, echo_server}, echo_server, [], []). > > %% public api > echo(Text) -> > gen_server:call(echo_server, {echo, Text}). > > %% crash this gen_server to test its supervisor > crash() -> > gen_server:call(echo_server, crash). > > %% behaviours > init(_Args) -> > {ok, none}. > handle_call({echo, Text}, _From, State) -> > {reply, Text, State}; > handle_call(crash, _From, State) -> > X=1, > {reply, X=2, State}. > handle_cast(_, State) -> > {noreply, State}. > > % -- end echo_server.erl -- % > > > > > echo_sup: > > % -- begin echo_sup.erl -- % > > -module(echo_sup). > -behaviour(supervisor). > -export([start_link/0]). > -export([init/1]). > > start_link() -> > supervisor:start_link({local, echo_sup}, echo_sup, []). > init(_Args) -> > {ok, {{one_for_one, 5, 60}, > [{echo_server, {echo_server, start_link, []}, > permanent, brutal_kill, worker, [echo_server]}]}}. > > > % -- end echo_sup.erl -- % > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bernie@REDACTED Sun May 9 03:06:54 2010 From: bernie@REDACTED (Bernard Duggan) Date: Sun, 09 May 2010 11:06:54 +1000 Subject: [erlang-questions] mnesia queries In-Reply-To: References: Message-ID: <4BE60AAE.1080602@m5net.com> Basically the answer is 'no', there isn't a much better way to do it than to drop an re: operation in the query ('re' is the new 'regex'), and yes this will be "slow" (for some definition of "slow"). The thing to remember is that mnesia isn't an SQL database, nor is it trying to be. Like every db system, it's built of compromises to be really good at some stuff (for example really fast lookup of key-value style data, distribution of that data across nodes so that you can do that read work on any node without needing any network traffic, storing arbitrary erlang terms rather than fixed-type data etc) at the expense of not being particularly good at other stuff. If you want it to be an SQL database, you're going to be disappointed - it isn't one. If, however, what you need is what mnesia provides then it's totally brilliant and leaves SQL systems looking fairly silly :) Cheers, Bernard On 9/05/2010 3:59 AM, Marcel Meyer wrote: > Wes, > I'm still digging into Erlang as well, but I would guess your predicate > function will have to do a regex on the value. > I hope the pros can suggest something better, as I read that Erlang's Regex > lib is awfully slow. > > Regards, > Marcel > > On 8 May 2010 10:54, Wes James wrote: > > >> I have been looking at mnesia and can do some simple quieries. Are >> there queries like in sql where you have: where field like '%data' - >> that is a "like" with the "%" operator? >> >> thx, >> >> -wes >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> From kenji.rikitake@REDACTED Sun May 9 04:35:31 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 9 May 2010 11:35:31 +0900 Subject: [erlang-questions] strip end off of string In-Reply-To: <77A1754B-3894-4641-A0B8-9D5E0787D946@cs.otago.ac.nz> References: <77A1754B-3894-4641-A0B8-9D5E0787D946@cs.otago.ac.nz> Message-ID: <20100509023531.GA38194@k2r.org> In the message <77A1754B-3894-4641-A0B8-9D5E0787D946@REDACTED> dated Fri, May 07, 2010 at 05:30:31PM +1200, Richard O'Keefe writes: > The bottom line is that lists can *only* be traversed from front > to back, because there are *only* forward pointers. Indeed. Searching a list: O(n) (where n is the number of the elements) Searching a red-black tree: O(log(n)) An example of slowness of a list operation: assignment and matching of multiple IPv4 or IPv6 addresses to a network interface on FreeBSD. Even though the address list for each interface is a TAILQ (doubly-linked list), the search time is still O(n). I've had a problem when n > 1000 or even n > 100. > One of the things I keep telling people is "STRINGS ARE WRONG". > For many purposes, a list of tokens is superior to a string. > (For most other purposes, some kind of tree is even better. > I've had spectacular speedups in C from using trees of > fragments instead of vectors of characters.) Preparing a fast-searchable data structures is critical. Just my JPY 2 thought, Kenji Rikitake From aj@REDACTED Sun May 9 06:08:50 2010 From: aj@REDACTED (AJ Heller) Date: Sat, 8 May 2010 21:08:50 -0700 Subject: [erlang-questions] beginning erlang/OTP - spawning gen_servers and supervisors In-Reply-To: <8F6C0C8C-FBB8-4403-B9DD-24A9D7CBEBE6@erlang.geek.nz> References: <8F6C0C8C-FBB8-4403-B9DD-24A9D7CBEBE6@erlang.geek.nz> Message-ID: Thank you, Geoff. That is the best explanation I've read of why the supervisor dies after being spawned in this way. Don't you find it odd, then, that a gen_server can be spawned the same way, and it continues to run after its parent process exits? On Sat, May 8, 2010 at 5:19 PM, Geoff Cant wrote: > If you do spawn(echo_sup, start_link, []), you'll spawn a process (A) that will spawn and link to an echo_sup supervisor process (B) which will spawn your gen_server (C). A will then exit because it has no more code to run (ala exit(normal)), causing B to exit (due to the link to A), causing C to exit (due to the link to B). > > You should either run echo_sup:start_link() at the shell (and then unlink the pid returned as otherwise mistakes you type in the shell will kill echo_sup) or you want to start echo_sup underneath an already running supervisor. (Or you want to write an application callback echo_app, and an echo.app file i.e. build an echo_server OTP application and have the application_controller deal with the top level of supervision). > > Cheers, > -Geoff > > On 2010-05-08, at 16:38 , AJ Heller wrote: > >> I am reading through the OTP documentation, stopping to work with and >> understand every piece as it is introduced. I've written a very simple >> gen_server and a simpler supervisor, and they both work as I expect >> them to, but I've had to stop there. I'd like to understand them each >> before I move on to building an application, but I'm confused about >> the behavior of the system when I spawn each of these pieces >> independently. >> >> (For reference, my gen_server (echo_server) and supervisor (echo_sup) >> modules are at the end of this post) >> >> I am able to spawn my simple gen_server from my erl shell using >> `spawn(echo_server, start_link, []).` I can then interact with it >> using its public api (`echo_server:echo(...)`). >> >> However, I cannot spawn a supervisor the same way. By calling >> `spawn(echo_sup, start_link, []).`, my supervisor and gen_server both >> appear to exit immediately after they have finished initializing. >> Everything looks fine until I try to interact with my gen_server and >> get >> >> "** exception exit: {noproc,{gen_server,call,[echo_server,{echo,hi}]}} >> ? ?in function ?gen_server:call/2" >> >> Why is it that I can spawn a gen_server but not a supervisor? Thank you much. >> >> >> --------------------------------------------------------------------------------------------------------- >> echo_server: >> >> % -- begin echo_server.erl -- % >> >> -module(echo_server). >> -behaviour(gen_server). >> -export([start_link/0]). >> -export([echo/1, crash/0]). >> -export([init/1, handle_call/3, handle_cast/2]). >> >> start_link() -> >> ? gen_server:start_link({local, echo_server}, echo_server, [], []). >> >> %% public api >> echo(Text) -> >> ? gen_server:call(echo_server, {echo, Text}). >> >> %% crash this gen_server to test its supervisor >> crash() -> >> ? gen_server:call(echo_server, crash). >> >> %% behaviours >> init(_Args) -> >> ? {ok, none}. >> handle_call({echo, Text}, _From, State) -> >> ? {reply, Text, State}; >> handle_call(crash, _From, State) -> >> ? X=1, >> ? {reply, X=2, State}. >> handle_cast(_, State) -> >> ? {noreply, State}. >> >> % -- end echo_server.erl -- % >> >> >> >> >> echo_sup: >> >> % -- begin echo_sup.erl -- % >> >> -module(echo_sup). >> -behaviour(supervisor). >> -export([start_link/0]). >> -export([init/1]). >> >> start_link() -> >> ? supervisor:start_link({local, echo_sup}, echo_sup, []). >> init(_Args) -> >> ? {ok, ?{{one_for_one, 5, 60}, >> ? ? ? ? ?[{echo_server, {echo_server, start_link, []}, >> ? ? ? ? ? ? ? ?permanent, brutal_kill, worker, [echo_server]}]}}. >> >> >> % -- end echo_sup.erl -- % >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kiszl@REDACTED Sun May 9 12:21:23 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sun, 09 May 2010 12:21:23 +0200 Subject: [erlang-questions] Inline If In-Reply-To: References: <4BE5DEEA.5070208@eonblast.com> <4BE5E4CC.6030600@tmit.bme.hu> <4BE5E7C2.10005@eonblast.com> <4BE5F1B3.3090708@tmit.bme.hu> Message-ID: <4BE68CA3.2000006@tmit.bme.hu> On 5/9/2010 11:05 AM, Masklinn wrote: > On 2010-05-09, at 01:20 , Zoltan Lajos Kis wrote: > >> I guess the ternary operator only makes sense in non-functional languages, where conditional statements do not have return values. >> Given that there was no ternary operator, you would have to duplicate the assignment part, rendering the code less readable: >> >> if (a) { x = b; } else { x = c; }; >> /* x = a ? b : c; */ >> >> In a functional language, such as Erlang, the these statements do have a return value, so the ternary-operator would only be a syntactic sugar: >> >> X = case A of true -> B; false -> C end. >> > For that precise case, using if would probably be a bit shorter (thought the second case might not be as readable): > > X = if A -> B; true -> C end. > > > I don't favor the "if" statement exactly because of the "true case". If I had to use if, I would do it this way: X = if A -> B; (not A) -> C end. From masklinn@REDACTED Sun May 9 11:05:00 2010 From: masklinn@REDACTED (Masklinn) Date: Sun, 9 May 2010 11:05:00 +0200 Subject: [erlang-questions] Inline If In-Reply-To: <4BE5F1B3.3090708@tmit.bme.hu> References: <4BE5DEEA.5070208@eonblast.com> <4BE5E4CC.6030600@tmit.bme.hu> <4BE5E7C2.10005@eonblast.com> <4BE5F1B3.3090708@tmit.bme.hu> Message-ID: On 2010-05-09, at 01:20 , Zoltan Lajos Kis wrote: > I guess the ternary operator only makes sense in non-functional languages, where conditional statements do not have return values. > Given that there was no ternary operator, you would have to duplicate the assignment part, rendering the code less readable: > > if (a) { x = b; } else { x = c; }; > /* x = a ? b : c; */ > > In a functional language, such as Erlang, the these statements do have a return value, so the ternary-operator would only be a syntactic sugar: > > X = case A of true -> B; false -> C end. For that precise case, using if would probably be a bit shorter (thought the second case might not be as readable): X = if A -> B; true -> C end. From alessandro.sivieri@REDACTED Sun May 9 12:39:38 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Sun, 9 May 2010 12:39:38 +0200 Subject: Supervisor, bridge and error messages Message-ID: Hi all, as a result of a previous discussion, I am now launching processes from a gen_server with supervisor_bridge, so I can have them under supervision. The following code is my bridge module: -module(my_process). -export([install/1, start/1, init/1, terminate/2]). install(F) -> Key = uuid:uuid(), ChildPid = proc_lib:spawn_link(fun() -> F() end), Params = {Key, {?MODULE, start, [ChildPid]}, temporary, infinity, supervisor, [?MODULE]}, {ok, _BridgePid} = supervisor:start_child(main_sup, Params), {Key, ChildPid}. start([ChildPid]) -> supervisor_bridge:start_link(?MODULE, [ChildPid]). init([ChildPid]) -> {ok, ChildPid, ChildPid}. terminate(_Reason, ChildPid) -> exit(ChildPid, kill). I need to create the non-otp process linked to the supervisor_bridge externally (in install()), instead of in init(), because I need its pid for communicating with it, and I have found that I cannot bring any term outside the init() function; so, this process is created outside and then linked inside the bridge itself (which should work, I have checked the supervisor_bridge sources). But then, when calling install() (with a valid fun as argument), the following report is created: ** Reason for termination == ** {{badmatch, {error, {{'EXIT', {function_clause, [{my_process,start,[<0.68.0>]}, {supervisor,do_start_child,2}, {supervisor,handle_start_child,2}, {supervisor,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]}}, {child,undefined,'17a10803-f064-4718-ae46-4a6d3c88415c', {my_process,start,[<0.68.0>]}, temporary,infinity,supervisor, [my_process]}}}}, [{my_process,install,1}, {my_peer,handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} Now, I have no idea from where the EXIT error is produced; I have added a print line in start(), before start_link, but it seems not to be executed; what am I doing wrong here? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From bernie@REDACTED Sun May 9 13:40:39 2010 From: bernie@REDACTED (Bernard Duggan) Date: Sun, 09 May 2010 21:40:39 +1000 Subject: [erlang-questions] beginning erlang/OTP - spawning gen_servers and supervisors In-Reply-To: References: <8F6C0C8C-FBB8-4403-B9DD-24A9D7CBEBE6@erlang.geek.nz> Message-ID: <4BE69F37.7030702@m5net.com> On 9/05/2010 2:08 PM, AJ Heller wrote: > Don't you find it odd, then, that a gen_server can be spawned the same > way, and it continues to run after its parent process exits? > It would be odd had Geoff's description been completely correct, but it was wrong in one crucial point: Linked processes don't automatically die when a process they are linked to exits with code 'normal'. That's why echo_sup doesn't exit when the spawning process exits. So why does the supervisor die? The internals of the supervisor module are in fact themselves implemented as a gen_server, but with process_flag(trap_exit, true) set. The result of this is that when the parent process dies, terminate() gets called (which doesn't happen when trap_exit is disabled) and the supervisor shuts down. It makes sense in the context of a supervisor, since a supervisor is spawned by its parent in a supervision tree - if it didn't die whenever its parent shutdown, whatever the reason, you'd have dangling "branches" of the tree. For what it's worth, I've been writing erlang for a couple of years now and am still only just getting completely to grips with the process shutdown and trap_exit semantics, particularly as they relate to gen_servers and supervisors. It all works great once you figure it out, but it's not exactly intuitive (which isn't meant as a criticisim - everything makes perfect sense from a design perspective once you puzzle it all out). Cheers, Bernard From rvirding@REDACTED Sun May 9 14:55:07 2010 From: rvirding@REDACTED (Robert Virding) Date: Sun, 9 May 2010 14:55:07 +0200 Subject: [erlang-questions] Inline If In-Reply-To: <4BE68CA3.2000006@tmit.bme.hu> References: <4BE5DEEA.5070208@eonblast.com> <4BE5E4CC.6030600@tmit.bme.hu> <4BE5E7C2.10005@eonblast.com> <4BE5F1B3.3090708@tmit.bme.hu> <4BE68CA3.2000006@tmit.bme.hu> Message-ID: On 9 May 2010 12:21, Zoltan Lajos Kis wrote: > On 5/9/2010 11:05 AM, Masklinn wrote: >> >> On 2010-05-09, at 01:20 , Zoltan Lajos Kis wrote: >> >>> >>> I guess the ternary operator only makes sense in non-functional >>> languages, where conditional statements do not have return values. >>> Given that there was no ternary operator, you would have to duplicate the >>> assignment part, rendering the code less readable: >>> >>> if (a) { x = b; } else { x = c; }; >>> /* x = a ? b : c; */ >>> >>> In a functional language, such as Erlang, the these statements do have a >>> return value, so the ternary-operator would only be a syntactic sugar: >>> >>> X = case A of true -> ?B; false -> ?C end. >>> >> >> For that precise case, using if would probably be a bit shorter (thought >> the second case might not be as readable): >> >> X = if A -> ?B; true -> ?C end. >> >> >> > > I don't favor the "if" statement exactly because of ?the "true case". If I > had to use if, I would do it this way: > > X = if A -> B; (not A) -> C end. You are thinking conventional 'if', view it rather as a (cond ...) with restricted tests or as a short form of case 1 of _ when ... -> ...; _ when ... -> ...; ... end (which it really is) and it all becomes clearer and more logical. :-) Robert From the.ajarn@REDACTED Sun May 9 17:05:18 2010 From: the.ajarn@REDACTED (Brentley Jones) Date: Sun, 9 May 2010 10:05:18 -0500 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4BE51604.8020907@eonblast.com> References: <4BE51604.8020907@eonblast.com> Message-ID: I have no answers, but I too would love to see what people say. - Brentley Jones On May 8, 2010, at 2:43 AM, Henning Diedrich wrote: > 1 > > I am trying to get a better picture of what the performance costs > are when mutating (collections of) complex structures. > > Most generally, if I have a list of structures and want to make one > change to one of the structures; and if I want to avoid ETS for the > implicit copying, what else should I NOT do to not run into the same > problem with a non-ETS approach, and how CAN I have as little > unnecessary internal copying of all those things that stay the same. > > I just ran into this post by J Louis while looking for information > > http://jlouisramblings.blogspot.com/2009/01/common-erlang-misconceptions.html > > It was giving some implicit answers about what results in internal > copying, what doesn't. And I am not talking about strings here. > > I wondered if there is a dedicated write up out there that explains > a little more to the point what the use of lists, sets, trees, ets > and mnesia implies in terms of copying, to better understand the > performance implications. With immutable variables, of course there > is always the hope that underneath, not much copying is actually > necessary, as I understand e.g. lists are benefiting hugely from, to > become O(1) when adding heads to them. So assumptions based on ALGOL > heritage experiences should fail. Which puts one on a quest like Max > Lapshin's http://erlang.2086793.n4.nabble.com/Help-with-storing-data-in-memory-td2119208.html#a2119208 > who is also really looking for a way to avoid implicit copying. > > I have been looking for clues in this direction all the time but > found them rather scarce in the docs. May be me. May be an idea to > put more focus on in erldocs, erlang.org etc? I am sure there are > links. Maybe they could be featured more prominently. I am sure > there are many people coming to Erlang with a deeply ingrained eye > on performance and would like to understand these implications > earlier in the learning experience. > > I mean things exactly like > > "changing status from x to y would copy the record, BUT the actual > record structure is only a set of pointers." by Ulf Wiger in his post > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/1342 (var names changed) > > 2 > > From that post I understand that the best performance should come > from a data structure that is as nested as possible, because that > will reduce the amount of pointers that are copied with every > mutation?! > > As the mutation copies not-changed data only on one level. That's a > very interesting implication that so far I haven't found spelled out > anywhere (but would have liked to learn up front, if true). I am > still not sure if it would hold across the board. Or is it wrong for > other reasons I am not aware of? (Or could this post maybe for any > strange reason be ... outdated? It's a decade old in June. Wow!) > > 3 > > Surveying the choices, does this imply I am to use a copy-heavy > approach if I want to have meaningful transaction support between > processes? Because the choice there is ets, dets or mnesia and they > all copy the entire structure everytime there is any access to it? > > 4 > > I would encourage a view that these kind of questions would be best > discussed quite early in the learning curve. Together with the > implications that immutable state has for message passing and how it > saves so much copying in that case. Or so I speculate. I am not > speaking about the error prevention aspects here. > > That said, thanks for all the efforts that are going into making the > Erlang docs ever better! Even Google today adopted the classic > format of the Erlang doc frame layout I realize. But seriously, > thanks! > > Henning > > From dmitriid@REDACTED Mon May 10 08:59:32 2010 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Mon, 10 May 2010 09:59:32 +0300 Subject: [erlang-questions] A historical questions about binaries In-Reply-To: <4BE5D457.9080605@eonblast.com> References: <4BE5D1C6.7020303@hyber.org> <4BE5D457.9080605@eonblast.com> Message-ID: <087EC76D-BC48-4400-883E-6372935BBC47@gmail.com> Indeed : )))) > This is scary. Erlang is not quite ancient just yet and the mysteries about its beginnings have already started to fade from human memory. > > Claes Wikstrom wrote: >> On 05/07/2010 01:43 PM, Dmitrii Dimandt wrote: >>> A friend of mine is digging deep into Erlang history and here's one question he hasn't found an answer to: >>> >>> At what time were binaries introduced in Erlang? I know that bit syntax appeared in 1999, it says so in the docs :) Were binaries introduced earlier? And what did they look like back then? >> >> Binaries came a bit earlier, I introduced them during an effort to separate the emulator proper >> from OS calls and then in particular the file system calls that were used to load code. >> Thus came drivers, the whole driver arch, and in particular the file driver that replied with binaries >> that could be fed to the code loading BIFs. >> >> My guess would be 1995-1996 >> >> /klacke Thank you wall for your replies! From rtrlists@REDACTED Mon May 10 10:31:40 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 10 May 2010 09:31:40 +0100 Subject: [erlang-questions] Supervisor, bridge and error messages In-Reply-To: References: Message-ID: On Sun, May 9, 2010 at 11:39 AM, Alessandro Sivieri < alessandro.sivieri@REDACTED> wrote: > Hi all, > > as a result of a previous discussion, I am now launching processes from a > gen_server with supervisor_bridge, so I can have them under supervision. > The following code is my bridge module: > > -module(my_process). > -export([install/1, start/1, init/1, terminate/2]). > install(F) -> > Key = uuid:uuid(), > ChildPid = proc_lib:spawn_link(fun() -> F() end), > Params = {Key, {?MODULE, start, [ChildPid]}, temporary, infinity, > supervisor, [?MODULE]}, > {ok, _BridgePid} = supervisor:start_child(main_sup, Params), > {Key, ChildPid}. > start([ChildPid]) -> > supervisor_bridge:start_link(?MODULE, [ChildPid]). > init([ChildPid]) -> > {ok, ChildPid, ChildPid}. > terminate(_Reason, ChildPid) -> > exit(ChildPid, kill). > > I need to create the non-otp process linked to the supervisor_bridge > externally (in install()), instead of in init(), because I need its pid for > communicating with it, and I have found that I cannot bring any term > outside > the init() function; so, this process is created outside and then linked > inside the bridge itself (which should work, I have checked the > supervisor_bridge sources). > But then, when calling install() (with a valid fun as argument), the > following report is created: > > ** Reason for termination == > ** {{badmatch, > {error, > {{'EXIT', > {function_clause, > [{my_process,start,[<0.68.0>]}, > {supervisor,do_start_child,2}, > {supervisor,handle_start_child,2}, > {supervisor,handle_call,3}, > {gen_server,handle_msg,5}, > {proc_lib,init_p_do_apply,3}]}}, > {child,undefined,'17a10803-f064-4718-ae46-4a6d3c88415c', > {my_process,start,[<0.68.0>]}, > temporary,infinity,supervisor, > [my_process]}}}}, > [{my_process,install,1}, > {my_peer,handle_call,3}, > {gen_server,handle_msg,5}, > {proc_lib,init_p_do_apply,3}]} > > Now, I have no idea from where the EXIT error is produced; I have added a > print line in start(), before start_link, but it seems not to be executed; > what am I doing wrong here? > > > Your child spec states: {?MODULE, start, [ChildPid]} as the Module, Function, Arglist to use. But your definition of start() expects a list to be passed in, not a simple pid. So, simply remove the [] brackets from your start/1 definition, I'd think. Oh, the error function_clause means that you were trying to invoke a function that exists, but where the arguments could not be matched to any clause. Robby From raimo+erlang-questions@REDACTED Mon May 10 13:59:33 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 10 May 2010 13:59:33 +0200 Subject: [erlang-questions] multicore performance fine grained concurrency In-Reply-To: References: <20100507072748.GA14799@erix.ericsson.se> Message-ID: <20100510115933.GA7117@erix.ericsson.se> On Fri, May 07, 2010 at 11:41:27AM -0600, Tony Arcieri wrote: > On Fri, May 7, 2010 at 1:27 AM, Raimo Niskanen < > raimo+erlang-questions@REDACTED > > wrote: > > > There are many different reasons why SMP, especially SMP benchmarks > > (as Kenneth explained in another mail in this thread) performs poorly. > > > > Wouldn't some kind of scheduler affinity for processes that send a lot of > messages to each other and don't consume a lot of CPU be helpful in cases > like this? That is part of the much bigger problem of how to schedule processes, and for the ring benchmark it is an obvious criterion, but for the general problem there are more (criterions) and to make a perfect in every situation scheduling algorighm is presumably impossible. We are of course aiming to get scheduling good enough with as little degenerated cases possible one day, but before that day it will become possible to control some aspects, such as processor affinity, by the application programmer, to aid specific applications. > > -- > Tony Arcieri > Medioh! A Kudelski Brand -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From alessandro.sivieri@REDACTED Mon May 10 16:12:30 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Mon, 10 May 2010 16:12:30 +0200 Subject: [erlang-questions] Supervisor, bridge and error messages In-Reply-To: References: Message-ID: 2010/5/10 Robert Raschke > Oh, the error function_clause means that you were trying to invoke a > function that exists, but where the arguments could not be matched to any > clause. > Ah yes, of course, I knew there was some stupid error somewhere... :) Anyway, now that aspect is solved, but there is one more interesting thing: in a situation like the one pictured here [1], where 78 and 81 are bridges and 77 and 80 are the two monitored processes (created with the code previously reported), if I kill one of those 4 processes, all the 4 ones are killed, while I thought that only the bridge (if killing its child) or the child (if killing the bridge) should die, while the others shouldn't be involved. Why this is not happening? The main supervisor policy is one_for_one. [1] http://yfrog.com/5jtreeip -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From pablo.platt@REDACTED Mon May 10 19:45:38 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 10 May 2010 10:45:38 -0700 (PDT) Subject: performance of re:replace/4 compared to plain code Message-ID: <126445.83704.qm@web112606.mail.gq1.yahoo.com> Hi, I'm building a driver for a database that should be fast. I need to escape <<".">> from binary strings that goes to the database with <<",">> I can do this with the re:replace/4 re:replace(<<"some.text.with.dots">>,<<"\\.">>,<<",">>,[{return,binary}]). I can also do it with a simple function: escape(Old) -> escape(Old, <<>>). escape(<<>>, New) -> New; escape_db(Old, New) -> <> = Old, case First of <<".">> -> escape(Rest, <>); _ -> escape(Rest, <>) end. The size of each binary string is <10 characters but there will be a lot of them processed all the time. Should I expect a difference in performance between the two ways? Which one should be faster and require less CPU? How can I benchmark it? Thanks From hynek@REDACTED Mon May 10 21:04:36 2010 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 10 May 2010 21:04:36 +0200 Subject: [erlang-questions] performance of re:replace/4 compared to plain code In-Reply-To: <126445.83704.qm@web112606.mail.gq1.yahoo.com> References: <126445.83704.qm@web112606.mail.gq1.yahoo.com> Message-ID: >From mine experience HiPE usually outperforms re module for this sort of task, but I would write simply: escape(Bin) -> << <<(case X of $.->$,; _->X end)>> || <> <= Bin >>. If performance really cares than you should use NIF but it is rarely reason. Most time you need just fast enough and above code used to be. And how benchmark? Just take real data sample. Duplicate to have enough amount if necessary and then measure. On Mon, May 10, 2010 at 7:45 PM, Pablo Platt wrote: > Hi, > > I'm building a driver for a database that should be fast. > I need to escape <<".">> from binary strings that goes to the database with <<",">> > > I can do this with the re:replace/4 > re:replace(<<"some.text.with.dots">>,<<"\\.">>,<<",">>,[{return,binary}]). > > I can also do it with a simple function: > escape(Old) -> > ? ?escape(Old, <<>>). > > escape(<<>>, New) -> > ? ?New; > > escape_db(Old, New) -> > ? ?<> = Old, > ? ?case First of > ? ? ? ?<<".">> -> > ? ? ? ? ? ?escape(Rest, <>); > ? ? ? ?_ -> > ? ? ? ? ? ?escape(Rest, <>) > ? ?end. > > The size of each binary string is <10 characters but there will be a lot of them processed all the time. > > Should I expect a difference in performance between the two ways? > Which one should be faster and require less CPU? > How can I benchmark it? > > Thanks > > > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From pablo.platt@REDACTED Mon May 10 21:15:00 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 10 May 2010 12:15:00 -0700 (PDT) Subject: [erlang-questions] performance of re:replace/4 compared to plain code In-Reply-To: References: <126445.83704.qm@web112606.mail.gq1.yahoo.com> Message-ID: <580916.95892.qm@web112614.mail.gq1.yahoo.com> I'll use you version. Thanks. ________________________________ From: Hynek Vychodil To: Pablo Platt Cc: Erlang Questions Sent: Mon, May 10, 2010 10:04:36 PM Subject: Re: [erlang-questions] performance of re:replace/4 compared to plain code >From mine experience HiPE usually outperforms re module for this sort of task, but I would write simply: escape(Bin) -> << <<(case X of $.->$,; _->X end)>> || <> <= Bin >>. If performance really cares than you should use NIF but it is rarely reason. Most time you need just fast enough and above code used to be. And how benchmark? Just take real data sample. Duplicate to have enough amount if necessary and then measure. On Mon, May 10, 2010 at 7:45 PM, Pablo Platt wrote: > Hi, > > I'm building a driver for a database that should be fast. > I need to escape <<".">> from binary strings that goes to the database with <<",">> > > I can do this with the re:replace/4 > re:replace(<<"some.text.with.dots">>,<<"\\.">>,<<",">>,[{return,binary}]). > > I can also do it with a simple function: > escape(Old) -> > escape(Old, <<>>). > > escape(<<>>, New) -> > New; > > escape_db(Old, New) -> > <> = Old, > case First of > <<".">> -> > escape(Rest, <>); > _ -> > escape(Rest, <>) > end. > > The size of each binary string is <10 characters but there will be a lot of them processed all the time. > > Should I expect a difference in performance between the two ways? > Which one should be faster and require less CPU? > How can I benchmark it? > > Thanks > > > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero! Try GoodData now for free: www.gooddata.com From mark.geib@REDACTED Tue May 11 00:57:35 2010 From: mark.geib@REDACTED (Mark Geib) Date: Mon, 10 May 2010 16:57:35 -0600 Subject: problem running appmon remotely via ssh Message-ID: <4BE88F5F.8070804@echostar.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am trying to run appmon:start() on a remote R12B5 node I connect via "ssh -X" and the display never opens. I have done this with R13Bx and it works find. Is there a way to make this work in R12x.?? Thanks, Mark. - -- Principal Engineer Cheyenne Software Engineering mark.geib@REDACTED / 35-215 PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvoj18ACgkQNMVrR9CRdv031ACfW9vifo9g7joO6Rn+hvCbr0b7 CTIAn0p0C0e0PV39EltQ2MZdjF9bGTXv =QREr -----END PGP SIGNATURE----- From bernie@REDACTED Tue May 11 03:58:19 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 11 May 2010 11:58:19 +1000 Subject: gen_server cleanup handling Message-ID: <4BE8B9BB.7070809@m5net.com> Hi list, This is something of a followup to my previous question about supervision trees. Basically we have a gen_server that, in the case of a standard app shutdown, needs to do some cleanup. A standard app shutdown seems to take the form of supervisors sending 'shutdown' exit messages to their children. Fine so far. The catch is that in order for the terminate() handler to be invoked on our server, we have to turn on trap_exit in every gen_server that needs to do cleanup. This seems non-ideal for a few reasons: * Elsewhere in the docs, we're cautioned against using trap_exit except when unavoidable (though I'm happy to accept it if this is one such case). * It means that we don't get automatic propagation of crashes from any worker processes we might spawn. Instead we have to write a handle_info({'EXIT'...) in every gen_server that might ever link to another process to ensure those crashes are propagated properly. This seems like it could be solved by instead spawning any worker processes as a child of our supervisor (which is what Garrett suggested we should do anyway) - if that's a good reason to set up things that way, I'm likewise happy to accept it and rearrange our code accordingly. * Most concerning, though, is the possibility of some library call creating a link to a temporary worker process (or any process for that matter) whose crash should propagate through us - in this case we'd still have to have the handle_info({'EXIT'...) setup as a catchall which seems like a fiddly, repetitive bit of code we'd rather avoid if possible. So what's the thinking about this? Am I missing something obvious? Should I just turn on trap_exit willy-nilly wherever I need shutdown cleanup? Should I just suck it up and write the 'EXIT' message handlers in all such gen_servers? Cheers, Bernard From mazen.harake@REDACTED Tue May 11 08:58:03 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 11 May 2010 09:58:03 +0300 Subject: [erlang-questions] gen_server cleanup handling In-Reply-To: <4BE8B9BB.7070809@m5net.com> References: <4BE8B9BB.7070809@m5net.com> Message-ID: <4BE8FFFB.4080406@erlang-solutions.com> Spawn your worker processes in a simple_one_for_one supervisor which is under your supervisor. This will propagate the exit to the worker processes. Use the gen_server under the first Sup to start a worker child in the simple_one_for_one Sup. In other words; Garrett seems to have gotten it right. This also eliminates point 3 because the crash will not propagate through the gen_server. I'm assuming from your setup that the gen_server doesn't care if the worker crashed or not before it finished, otherwise you might as well just trap exits and handle all the exit messages anyway without a supervisor. Hope this makes sense; it is morning here and my coffee cup is still full (didn't have a chance to drink it yet :)) Good luck /Mazen On 11/05/2010 04:58, Bernard Duggan wrote: > Hi list, > This is something of a followup to my previous question about > supervision trees. Basically we have a gen_server that, in the case > of a standard app shutdown, needs to do some cleanup. A standard app > shutdown seems to take the form of supervisors sending 'shutdown' exit > messages to their children. Fine so far. The catch is that in order > for the terminate() handler to be invoked on our server, we have to > turn on trap_exit in every gen_server that needs to do cleanup. This > seems non-ideal for a few reasons: > > * Elsewhere in the docs, we're cautioned against using trap_exit > except when unavoidable (though I'm happy to accept it if this is one > such case). > > * It means that we don't get automatic propagation of crashes from any > worker processes we might spawn. Instead we have to write a > handle_info({'EXIT'...) in every gen_server that might ever link to > another process to ensure those crashes are propagated properly. This > seems like it could be solved by instead spawning any worker processes > as a child of our supervisor (which is what Garrett suggested we > should do anyway) - if that's a good reason to set up things that way, > I'm likewise happy to accept it and rearrange our code accordingly. > > * Most concerning, though, is the possibility of some library call > creating a link to a temporary worker process (or any process for that > matter) whose crash should propagate through us - in this case we'd > still have to have the handle_info({'EXIT'...) setup as a catchall > which seems like a fiddly, repetitive bit of code we'd rather avoid if > possible. > > So what's the thinking about this? Am I missing something obvious? > Should I just turn on trap_exit willy-nilly wherever I need shutdown > cleanup? Should I just suck it up and write the 'EXIT' message > handlers in all such gen_servers? > > Cheers, > > Bernard > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From alex.arnon@REDACTED Tue May 11 10:03:28 2010 From: alex.arnon@REDACTED (Alex Arnon) Date: Tue, 11 May 2010 11:03:28 +0300 Subject: [erlang-questions] Klarna/Kreditor + Sequoia In-Reply-To: <520948.95593.qm@web111416.mail.gq1.yahoo.com> References: <520948.95593.qm@web111416.mail.gq1.yahoo.com> Message-ID: Congrats to all involved, well done!!! On Thu, May 6, 2010 at 11:50 PM, Thomas Lindgren wrote: > > I haven't seen it mentioned on the list, but Klarna/Kreditor (who do quite > a bit of erlang) just got an investment by Sequoia Capital and are getting > Michael Moritz on the board of directors to boot. Could we enthusiasts at > this point say erlang has arrived? :-) Congratulations, guys. Keep at it. > > http://www.klarna.se/en/press/258-sequoia-capital-investerar-i-klarna > > Best, > Thomas > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From wiener.guy@REDACTED Tue May 11 11:06:08 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Tue, 11 May 2010 12:06:08 +0300 Subject: Looking for Nitrogen 2 Windows executables Message-ID: Hello everyone, I need windows binaries for Nitrogen 2 rather urgently, and I don't have a complete development environment on my windows machine (It's a lab computer, not my-own). Can someone help me out on this? I'll take the binaries as-is, no warranties. Preferably the Yaws package, but not a must. If someone has such binaries, please answer this mail directly to me, not the list. Thanks in advance, Guy Wiener. From max.lapshin@REDACTED Tue May 11 11:53:46 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 11 May 2010 13:53:46 +0400 Subject: Current state of packaged modules: net.rtmp.decoder Message-ID: Hi, I can't understand why is not recommended now to use packages of modules. What is bad in using dotted names? What is the future of this feature: it will be developed or it will be deprecated? From caio.ariede@REDACTED Tue May 11 15:09:02 2010 From: caio.ariede@REDACTED (caio ariede) Date: Tue, 11 May 2010 10:09:02 -0300 Subject: Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform Message-ID: Both platforms give me the same error, just alterning the text specifying the platform ("x86_64-pc-linux-gnu") and the path, on the error below: 1> wx:new(). =ERROR REPORT==== 11-May-2010::10:01:54 === WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: /usr/lib/erlang/lib/wx-0.98.2/priv ** exception error: {load_driver,"No driver found"} in function wxe_server:start/0 in call from wx:new/1 Caio Ariede http://caioariede.com/ From pablo.platt@REDACTED Tue May 11 14:40:41 2010 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 11 May 2010 05:40:41 -0700 (PDT) Subject: implications of keeping many erlang terms in the state of a gen_server Message-ID: <851708.69523.qm@web112608.mail.gq1.yahoo.com> Hi, I'm trying to understand if its ok to have a large number of erlang terms in the state of a gen_server. I'll have ~10K gen_servers running each with a state, a record, with several erlang terms. The state will also have ~5 lists each with ~100 erlang terms. The gen_server will need to add items to the head of the list and to pass the whole list but not random access. The gen_server will be accessed every 1 minute so I don't think hibernate should be used. What are the costs of keeping a lot of erlang terms in the state? Are there cpu or memory costs except for the actual size of the elements? What are the costs of garbage collection and how often does it happens? Are there costs every time the gen_server is being called? Is there a difference between passing the whole state around in functions (in the same processes) and just passing a small part of the state? Is there a difference between changing the state several times in the same function and changing it once? Thanks From zeno490@REDACTED Tue May 11 16:51:38 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Tue, 11 May 2010 10:51:38 -0400 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: I would like to know as well as I enjoy working with them. Simplifies naming greatly when one isn't afraid of naming collisions. Moreover, I suspect that people that do encounter collisions, will use a prefix/suffix to deal with it, in a similar way to prefixing package name with '.'. Erlang would benefit greatly from a stable and officially supported namespace/package system for modules. IMO, I've been quite happy with the current state of things with package prefix. On Tue, May 11, 2010 at 5:53 AM, Max Lapshin wrote: > Hi, I can't understand why is not recommended now to use packages of > modules. What is bad in > using dotted names? What is the future of this feature: it will be > developed or it will be deprecated? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alessandro.sivieri@REDACTED Tue May 11 17:26:35 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Tue, 11 May 2010 17:26:35 +0200 Subject: [erlang-questions] Supervisor, bridge and error messages In-Reply-To: References: Message-ID: Ok, solved, it was my fault in creating children processes; new question: it seems that when starting a third bridge, the first one is killed (without a reason, it seems); here [1] there is the crash report, but I cannot find a good reason for this... there are the first two bridges creation, then the first bridge crash when I launched the third one. [1] http://pastebin.com/8Kw2JEL0 -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From MARTIN.LOGAN@REDACTED Tue May 11 18:15:21 2010 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Tue, 11 May 2010 11:15:21 -0500 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: I think adding package support in fully would be quite an undertaking. So many of the OTP tools don't understand them and I am not entirely sure that making it so would be entirely trivial. That said, I agree that having them would be a benefit though I have never run into a conflict simply using the _ convention. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Nicholas Frechette Sent: Tuesday, May 11, 2010 9:52 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Current state of packaged modules: net.rtmp.decoder I would like to know as well as I enjoy working with them. Simplifies naming greatly when one isn't afraid of naming collisions. Moreover, I suspect that people that do encounter collisions, will use a prefix/suffix to deal with it, in a similar way to prefixing package name with '.'. Erlang would benefit greatly from a stable and officially supported namespace/package system for modules. IMO, I've been quite happy with the current state of things with package prefix. On Tue, May 11, 2010 at 5:53 AM, Max Lapshin wrote: > Hi, I can't understand why is not recommended now to use packages of > modules. What is bad in > using dotted names? What is the future of this feature: it will be > developed or it will be deprecated? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From prikrutil@REDACTED Tue May 11 19:32:33 2010 From: prikrutil@REDACTED (Sergey Samokhin) Date: Tue, 11 May 2010 21:32:33 +0400 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> Message-ID: > Having too many meanings for the same symbol is too confusing and > error-prone. ?I didn't use "=" because that already means "match". You are right in that '=' is used here and there in Erlang. We use '=' to bind variable name to its value: Var = 42. And also to pattern match on value: 42 = Var. Why use yet another symbol to do these usual things? I find the following syntax most convenient: #{key = "val"} It's like syntax for records, but without the record name. Everyone who see this will say: Oh, it looks like records with no record type specified. It should be something like an anonymous record. And he will be almost right. -- Sergey Samokhin From pfisher@REDACTED Tue May 11 20:02:07 2010 From: pfisher@REDACTED (Paul Fisher) Date: Tue, 11 May 2010 13:02:07 -0500 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> Message-ID: <1273600927.2995.0.camel@pfisher-laptop> +1 On Tue, 2010-05-11 at 12:32 -0500, Sergey Samokhin wrote: > > Having too many meanings for the same symbol is too confusing and > > error-prone. I didn't use "=" because that already means "match". > > You are right in that '=' is used here and there in Erlang. > > We use '=' to bind variable name to its value: Var = 42. > And also to pattern match on value: 42 = Var. > > Why use yet another symbol to do these usual things? > > I find the following syntax most convenient: > > #{key = "val"} > > It's like syntax for records, but without the record name. > > Everyone who see this will say: Oh, it looks like records with no > record type specified. It should be something like an anonymous > record. And he will be almost right. > From kagato@REDACTED Tue May 11 20:22:31 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 11 May 2010 11:22:31 -0700 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: <922E7046-BE65-432F-B3D1-3B94F91615C7@souja.net> +1 Sent from my iPhone On May 11, 2010, at 7:51 AM, Nicholas Frechette wrote: > I would like to know as well as I enjoy working with them. > Simplifies naming greatly when one isn't afraid of naming collisions. > Moreover, I suspect that people that do encounter collisions, will > use a > prefix/suffix to deal with it, in a similar way to prefixing package > name > with '.'. > > Erlang would benefit greatly from a stable and officially supported > namespace/package system for modules. IMO, I've been quite happy > with the > current state of things with package prefix. > > On Tue, May 11, 2010 at 5:53 AM, Max Lapshin > wrote: > >> Hi, I can't understand why is not recommended now to use packages of >> modules. What is bad in >> using dotted names? What is the future of this feature: it will be >> developed or it will be deprecated? >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From kagato@REDACTED Tue May 11 20:26:11 2010 From: kagato@REDACTED (Jayson Vantuyl) Date: Tue, 11 May 2010 11:26:11 -0700 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> Message-ID: <38E6AF05-AAB0-4625-AC94-204C6623B75D@souja.net> I would note that #{...} is used for string interpolation in Ruby. Ironically, I generate some Erlang from Ruby, so this would make for interesting escaping for me. That said, I think this is a fantastic syntax. Sent from my iPhone On May 11, 2010, at 10:32 AM, Sergey Samokhin wrote: >> Having too many meanings for the same symbol is too confusing and >> error-prone. I didn't use "=" because that already means "match". > > You are right in that '=' is used here and there in Erlang. > > We use '=' to bind variable name to its value: Var = 42. > And also to pattern match on value: 42 = Var. > > Why use yet another symbol to do these usual things? > > I find the following syntax most convenient: > > #{key = "val"} > > It's like syntax for records, but without the record name. > > Everyone who see this will say: Oh, it looks like records with no > record type specified. It should be something like an anonymous > record. And he will be almost right. > > -- > Sergey Samokhin > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bernie@REDACTED Wed May 12 02:47:15 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 12 May 2010 10:47:15 +1000 Subject: [erlang-questions] gen_server cleanup handling In-Reply-To: <4BE8FFFB.4080406@erlang-solutions.com> References: <4BE8B9BB.7070809@m5net.com> <4BE8FFFB.4080406@erlang-solutions.com> Message-ID: <4BE9FA93.4080709@m5net.com> Hi Mazen, Our setup kind of has the opposite requirement - if either the worker or the gen_server that spawned it crash, both should crash out (sorry, that probably wasn't very clear in my initial post). That's why initially a straight spawn_link to create the worker looked so attractive - errors propagate between the gen_server and worker, both are cleaned up, and the gen_server's supervisor just restarts the gen_server (which is what we want). That, though, would require us to handle the EXIT message from the worker in the gen_server if the latter turned on trap_exit (which it seems like it needs to). It's not a huge problem, just seems like something we should be able to avoid. The alternative we thought of (that I forgot to mention) is that this whole mess seems like it could be neatly avoided if supervisors also knew how to shutdown their child - I'm thinking an optional extra function in the child_spec that is called before the exit(Child, shutdown) call is made. That would allow the gen_server to do its cleanup properly and exit with 'stop'. There may be a very good (or historical or both) reason that that doesn't exist, but it seems to me to be the 'right' way to solve the problem...maybe. Cheers, Bernard On 11/05/10 16:58, Mazen Harake wrote: > Spawn your worker processes in a simple_one_for_one supervisor which is > under your supervisor. This will propagate the exit to the worker > processes. Use the gen_server under the first Sup to start a worker > child in the simple_one_for_one Sup. > > In other words; Garrett seems to have gotten it right. > > This also eliminates point 3 because the crash will not propagate > through the gen_server. > > I'm assuming from your setup that the gen_server doesn't care if the > worker crashed or not before it finished, otherwise you might as well > just trap exits and handle all the exit messages anyway without a > supervisor. > > Hope this makes sense; it is morning here and my coffee cup is still > full (didn't have a chance to drink it yet :)) > > Good luck > > /Mazen > > On 11/05/2010 04:58, Bernard Duggan wrote: > >> Hi list, >> This is something of a followup to my previous question about >> supervision trees. Basically we have a gen_server that, in the case >> of a standard app shutdown, needs to do some cleanup. A standard app >> shutdown seems to take the form of supervisors sending 'shutdown' exit >> messages to their children. Fine so far. The catch is that in order >> for the terminate() handler to be invoked on our server, we have to >> turn on trap_exit in every gen_server that needs to do cleanup. This >> seems non-ideal for a few reasons: >> >> * Elsewhere in the docs, we're cautioned against using trap_exit >> except when unavoidable (though I'm happy to accept it if this is one >> such case). >> >> * It means that we don't get automatic propagation of crashes from any >> worker processes we might spawn. Instead we have to write a >> handle_info({'EXIT'...) in every gen_server that might ever link to >> another process to ensure those crashes are propagated properly. This >> seems like it could be solved by instead spawning any worker processes >> as a child of our supervisor (which is what Garrett suggested we >> should do anyway) - if that's a good reason to set up things that way, >> I'm likewise happy to accept it and rearrange our code accordingly. >> >> * Most concerning, though, is the possibility of some library call >> creating a link to a temporary worker process (or any process for that >> matter) whose crash should propagate through us - in this case we'd >> still have to have the handle_info({'EXIT'...) setup as a catchall >> which seems like a fiddly, repetitive bit of code we'd rather avoid if >> possible. >> >> So what's the thinking about this? Am I missing something obvious? >> Should I just turn on trap_exit willy-nilly wherever I need shutdown >> cleanup? Should I just suck it up and write the 'EXIT' message >> handlers in all such gen_servers? >> >> Cheers, >> >> Bernard >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > From dmurray@REDACTED Wed May 12 03:21:17 2010 From: dmurray@REDACTED (David N Murray) Date: Tue, 11 May 2010 21:21:17 -0400 (EDT) Subject: [OT] This is sad Message-ID: http://blog.reddit.com/2010/05/reddits-may-2010-state-of-servers.html "After finally recovering from the Cassandra failure and preparing to head off for some much-needed sleep, our internal message bus (rabbitmq) died, which added about an hour to the downtime. It dies like this pretty often at 2am or at other especially bad times. Usually it doesn't cause any data-loss, just sleep-loss (its queues are persisted and the apps just build up their own queues until it comes back up), but in this case it decided to crash in a way that corrupted its database of persisted queues beyond repair. rabbitmq accounts for the only unrecoverable data-loss incurred, which was about 400 votes. As far as we can tell, these were entirely unlinked events. Coincidentally, rabbitmq crashed twice more that day and a few more times into the weekend. For now we've upgraded to the latest version of Erlang (rabbitmq is written in Erlang) since R13B-4 is rumoured to have significantly better memory management which can act as a temporary stopgap for the apparent reasons for some of the crashes, but not all of them. Things have improved thus far, but replacing rabbitmq is at the top end of our extremely long list of things to do." Not good publicity. :-( From ok@REDACTED Wed May 12 05:52:50 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 12 May 2010 15:52:50 +1200 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: On May 11, 2010, at 9:53 PM, Max Lapshin wrote: > Hi, I can't understand why is not recommended now to use packages of > modules. There are several important things here. (1) Erlang allows dots in atoms, always has. foo.bar.ugh and foo@REDACTED@ugh and foo_bar_ugh are just plain atoms. They are the _same_ atoms as 'foo.bar.ugh', 'foo@REDACTED@ugh', and 'foo_bar_ugh'. This does _not_ apply to Java, where foo_bar is one identifier but foo.bar is not. (2) Historically, the module system treated all atoms alike. There was no connection between foo.bar and bar any more than there was any connection between foo@REDACTED and bar or between foo_bar and bar. In the same way, Java sees no connection between a class called Foo_Bar and a class called Bar. One of the fundamental concepts in computer programming is alpha-conversion (the idea that names are *arbitrary* labels). It is generally a bad idea to violate this. (3) Formerly, in -module(fred), a call jill:bucket(56) was a call to the module 'jill'. Change the -module directive to -module(demo.fred) and that call now refers to the module 'demo.jill'. This is fine when you want it to happen, but if your code referred to existing OTP modules such as 'lists', well hello demo.lists breakage. You _can_ refer to top level modules using ''.lists or .lists, but that's an incompatible change. (4) It's really not clear how all of this works *dynamically*. Before dotted module names got special treatment, jill:bucket(56), %A M = jill, M:bucket(56), %B apply(jill, bucket, [56]), %C 'my.module':'my.apply'(jill, bucket, [56]) %D were all equivalent, where -module('my.module'). -export(['my.apply'/3]). my.apply(M, F, A) -> apply(M, F, A). The abbreviation kluge obviously works for %A. It could be made to work for %B by using different (slower!) code like M = jill, (fix_mod(M, 'my.')):bucket(56), where fix_mod(M, C) -> case has_dots(M) of true -> M ; false -> atom_append(C, M) end. It's easy to make %C work when the first argument is a constant, like %A, and when the first argument is not a constant, the same technique used for %B will work. But how do you make %D work? (5) There are no actual packages. There are directories where modules with the same prefix will be sought, but it is not the case that all files with the same prefix must be in the same directory. A "package" is nothing but a bunch of modules with a common prefix to their names; there are no package properties. (6) Package names are back to front. This has been explained repeatedly in this mailing list. They are like absolute file names, not like relative ones. If I need the same package in two places in the package hierarchy, I can't do it. I can't move a package up or down in the package hierarchy without surgery on every file. I cannot make a *component* that my customers can place anywhere they want in the package hierarchy. And if I can't do *that*, what the heck is the package system *for*? So: - dotted module names was not a new thing - mapping dotted names to the file system as described in http://www.erlang.se/publications/packages.html obviously has its uses, but this could be done with any character, including @, or indeed by other means entirely (like Eiffel's LACE) - the scheme leads to increased work when adopting dotted names and makes renaming packages difficult and fragile - it does not seem that dynamic behaviour can match the static abbreviation facility without increased costs (when known functions/forms are involved) or at all (when user defined functions are involved) > What is bad in > using dotted names? What is the future of this feature: it will be > developed or it will be deprecated? It should be taken out and shot. *Some* sort of module connection/configuration facility is needed, but I suggest as a design criterion that it should be possible to move an entire "flotilla" of modules in the package name space by changing *ONE* line in some file. From ok@REDACTED Wed May 12 06:30:46 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 12 May 2010 16:30:46 +1200 Subject: [erlang-questions] Re: Special syntax for dictionaries In-Reply-To: References: <88d94133-6462-405d-a73b-b6ea0d4fdbc4@o11g2000yqj.googlegroups.com> <9A133B38-60AF-4B1B-B73E-F84450A5D9CE@cs.otago.ac.nz> Message-ID: <80A27372-88A5-48A4-96C3-4BE0C5FA0542@cs.otago.ac.nz> On May 12, 2010, at 5:32 AM, Sergey Samokhin wrote: >> Having too many meanings for the same symbol is too confusing and >> error-prone. I didn't use "=" because that already means "match". > > You are right in that '=' is used here and there in Erlang. > > We use '=' to bind variable name to its value: Var = 42. > And also to pattern match on value: 42 = Var. No, those are the SAME thing. If you exclude records (which is precisely what I want to do), = means ONE thing. > > Why use yet another symbol to do these usual things? Nobody is suggesting any such thing. In my frames proposal, = remains the one and only symbol for pattern matching. But writing a maplet DOES NOT INVOLVE ANY PATTERN MATCHING! > > I find the following syntax most convenient: > > #{key = "val"} But that uses = for something which is in NO WAY the same as binding a variable name to its value (the thing on the left is NOT a variable) nor is it even close to being a pattern match on value ('key' is not the same as "val"). So you are saying that the most convenient syntax is one that uses the pattern matching symbol for something that not only is not pattern matching but isn't even related! That makes no more sense than using the addition operator would. In an expression <{key ~ Var = "val"}> the = on the right *IS* a pattern match and the ~ on the left *ISN'T*. That is why it would be stupid to make them the same symbol. > > It's like syntax for records, but without the record name. But the frames proposal allows something that looks very much like a record name: . Frames *aren't* records (because they are a run-time construct) even though they have many of the same parts as records (which are a purely compile-time construct). Because of the difference in semantics, it would be disastrous to give them the same syntax. #thingy{whatsit = Who_Jackie} is a record. is a frame. If they looked the same, how would you know what your program meant? In converting code to use frames, I found it *essential* to be able to tell which parts had been changed and which had not. This is not a defence of <{~}> . There may well be better symbols around (although in the case of ~ I have tried quite hard to find one and failed). It's an argument against copying records syntax slavishly. We *NEED* frame syntax to be *different* from record syntax because the difference in semantics means that we need to be *certain* which one we're dealing with. And the mistake (to put it kindly) of abusing = for something that is not matching is something we need to put behind us as quickly as we can. > From erlang@REDACTED Wed May 12 09:31:00 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 12 May 2010 09:31:00 +0200 Subject: Classification of concurrency patterns Message-ID: I'm interested in trying to classify and name a number of different parallelization strategies. We often use expressions like MPC "message passing concurrency" and SPC "shared state concurrency" this is not what I'm interested in. I'm interested in classifying and naming the strategies we use to write parallel code. We could *implement* these strategies using MPC or SSC. I've chosen my own names for these below, I have no idea if there are standard names for these classifications, this is what I've called them: 1) Divide and conquer concurrency 2) Pipeline concurrency 3) Map reduce concurrency 4) Identical Job concurrency 5) Grid concurrency 6) Job queue concurrency They mean the following: 1) Divide and conquer concurrency Divide the problem into K jobs which can be performed in parallel. Perform the jobs Recombine the results 2) Pipeline concurrency Divide the problem into K jobs that can be performed in a pipeline. The output of the first job must be the input of the second and so on. Run all the steps in parallel 3) Map reduce concurrency This is similar to 1). Only the recombination step involves merging results that have the same Key. 4) Identical Job concurrency Here the problem does not have to be split into K parts, we already have N identical jobs to do (identical means jobs that will consume similar resources) There is no recombination step. There are N results. 5) Grid concurrency Divide the problem into N*M identical jobs that can be solved on a NxM grid of processors. Each processor can only talk to its direct neighbors 6) Job queue concurrency Divide the problem into a set of named workers (who do jobs). Each worker has an input job queue. Workers consume jobs from their job queues and send the result to other workers. Note: [1] Paul Morrison calls this flow based programming [2] This is AMQP ---- That was my first attempt at naming and classifying these patterns. I'd be interested to know alternative names for the same things, and names of systems that employ the above strategies, and of course missing strategies. If we could name and classify these things then we could start writing gen_divide_and_conquer gen_pipe etc. /Joe From vladdu55@REDACTED Wed May 12 12:35:09 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 12 May 2010 12:35:09 +0200 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: Hello Joe, On Wed, May 12, 2010 at 09:31, Joe Armstrong wrote: > I'm interested in trying to classify and name a number of different > parallelization strategies. > 1) Divide and conquer concurrency > 2) Pipeline concurrency > 3) Map reduce concurrency > 4) Identical Job concurrency > 5) Grid concurrency > 6) Job queue concurrency IMHO, #1, #3 and #4 are the same thing, differently parametrized. Do you think it is important to get them separate identities? Regarding #5, a more general description would involve a grid that is not restricted to two dimensions. I think I remember seeing papers about 3D grids. #6 can be seen as a generalization of #2, where in the pipeline the "next" worker is hardcoded and the structure is linear. A pipeline may or may not use buffering queues between stages. The way I see it, there are two main categories: * dividing the data set into smaller chunks and processing them in parallel (#1,3,4) * dividing the work into specialized chunks and letting the data flow to next chunk (#2,6) These two can also be combined in several ways, giving for example #5 (usually each processor has its own chunk of input data, but data can flow to neighboring processors). Or each processing unit in #1 could involve a pipeline. It depends on the purpose of defining these categories whether a shorter, more general list or a more detailed one is required. best regards, Vlad > 1) Divide and conquer concurrency > > Divide the problem into K jobs which can be performed in parallel. > Perform the jobs > Recombine the results > > 2) Pipeline concurrency > > Divide the problem into K jobs that can be performed in a pipeline. > The output of the first job must be the input of the second and so on. > Run all the steps in parallel > > 3) Map reduce concurrency > > ? ?This is similar to 1). Only the recombination step involves > merging results that > ? have the same Key. > > 4) Identical Job concurrency > > ? ?Here the problem does not have to be split into K parts, we > already have N identical > ? ?jobs to do (identical means jobs that will consume similar resources) > > ? There is no recombination step. There are N results. > > 5) Grid concurrency > > ? ?Divide the problem into N*M identical jobs that can be solved on a > NxM grid of processors. > ? ?Each processor can only talk to its direct neighbors > > 6) Job queue concurrency > > ? ?Divide the problem into a set of named workers (who do jobs). Each > worker has an input > ? ?job queue. Workers consume jobs from their job queues and send the > result to other workers. > > ? ?Note: ?[1] Paul Morrison calls this flow based programming From dmurray@REDACTED Wed May 12 14:18:08 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 12 May 2010 08:18:08 -0400 (EDT) Subject: [erlang-questions] TCP receive in gen_server In-Reply-To: References: Message-ID: On May 8, Steve Vinoski scribed: > > Are you setting {active, once} on the socket? Or {active, true}? > Either will work but the {active, once} setting is preferred since it > will keep the writer from overrunning the reader. > > Pass {active, once} in your options list when you connect the socket. > Then, within your handle_info clause that receives the tcp messages, > call inet:setopts(Sock, [{active, once}]) before returning in order to > set up the socket for the next message. > I modified my code to connect as: case gen_tcp:connect(Host, Port, [list, inet, {active, once}, {exit_on_close, true}, {nodelay, true}, {packet, raw}, {recbuf, 524288}]) of and receive as: handle_info({tcp, Socket, Data}, State) -> % process data via read_pkt() inet:setopts(Socket, [{active, once}]), {noreply, NewState}; I'm still sending with another process. I'm still seeing the same problem. No data from the socket is sent to handle_info unless I first send something to the socket, and then, only one packet/message from the server is delivered at a time, even though there are at least 3 messages pending: 05/12:07:58:48: ib:connect: Connected 05/12:07:58:48: ibr:process_data.neg_time: ServerVersion=45 ServerTime=20100512 07:58:48 EST 05/12:07:58:48: ibr:read_pkt: Dispatch: [{msg_id,next_valid_id},{version,1},{orderId,3230}] 05/12:07:58:49: ibr:read_pkt: Dispatch: [{msg_id,open_order_end},{version,1}] (the above messages arrive as a single packet in response to me sending a client id as part of the connect/negotiate process.) 05/12:07:59:02: ibs:bld_send: [{pkt_id,49},{version,1}] %% I requested current time 05/12:07:59:02: ibr:read_pkt: Dispatch: [{msg_id,err_msg}, {version,2}, {id,-1}, {errorCode,2104}, {errorMsg,"Market data farm connection is OK:usfuture"}] that message was sitting in buffer 05/12:07:59:14: ibs:bld_send: [{pkt_id,49},{version,1}] %% again 05/12:07:59:14: ibr:read_pkt: Dispatch: [{msg_id,err_msg}, {version,2}, {id,-1}, {errorCode,2104}, {errorMsg,"Market data farm connection is OK:usfarm"}] so was this one 05/12:07:59:22: ibs:bld_send: [{pkt_id,49},{version,1}] %% and again 05/12:07:59:22: ibr:read_pkt: Dispatch: [{msg_id,err_msg}, {version,2}, {id,-1}, {errorCode,2107}, {errorMsg,"HMDS data farm connection is inactive but should be available upon demand.ushmds2a"}] and this one 05/12:07:59:33: ibs:bld_send: [{pkt_id,49},{version,1}] 05/12:07:59:33: ibr:read_pkt: Dispatch: [{msg_id,current_time},{version,1},{time,1273665541}] now I finally get a reply to the first request made at 59:02. Any thoughts would be appreciated. TIA, Dave From dmurray@REDACTED Wed May 12 14:33:58 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 12 May 2010 08:33:58 -0400 (EDT) Subject: [erlang-questions] TCP receive in gen_server In-Reply-To: References: Message-ID: On May 12, David N Murray scribed: > I'm still seeing the same problem. No data from the socket is sent to > handle_info unless I first send something to the socket, and then, only > one packet/message from the server is delivered at a time, even though > there are at least 3 messages pending: I figured it out. I had to call inet:setopts(Socket, [{active, true}]) after the gen_tcp:send/2, too. Thanks, Dave From alessandro.sivieri@REDACTED Wed May 12 15:55:32 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 12 May 2010 15:55:32 +0200 Subject: [erlang-questions] Supervisor, bridge and error messages In-Reply-To: References: Message-ID: Solved this one, too: only two children were allowed because I was reloading the binary code of the module executed, in each spawn, for other reasons, and the older child was killed (obviously). Thanks anyway! -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From rumata-estor@REDACTED Wed May 12 16:48:25 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Wed, 12 May 2010 18:48:25 +0400 Subject: Timeouted gen_server:call Message-ID: <4BEABFB9.5060103@nm.ru> I have a gen_server and many clients. Clients send request to the server with gen_server:call(Server, Msg, Timeout) and when timeout occurs they ask server to abort appropriate reply. But server may reply right before it receives abort message and I may get trash in client's mailbox. I'm not sure that client can throw away this trash itself. So it is possible for client to fill his message queue with lots of such messages. I may not let client crash. Neither I don't want to spawn new process for every such request. Looking through gen module I found that wait_resp*/3 could tell me Mref with exit({timeout, Mref}). Then I could use gen:call() and catch that Mref. But it currently doesn't offer such information. So, I'd like to know how more experienced programmers workaround this problem. Code like this: -export([do_call/4, handle_call/3, handle_info/2]). do_call(Server, MsgId, Msg, Timeout) -> try gen_server:call(Server, {process, MsgId, Msg}, Timeout) of Response -> Response catch exit:{timeout, Reason} -> %% this is a place when server can reply us Server ! {abort, MsgId}, %% here I'd like to cleanup message queue but I don't know Mref error end. handle_call({process, MsgId, Msg}, From, State) -> start_processing(MsgId, Msg), {noreply, State#state{ids=[{MsgId, From} | State#state.ids]}}. handle_info({ready, MsgId, Response}, State) -> case get_from(MsgId, State) of undefined -> {noreply, State}; {From, NewState} -> gen_server:reply(From, Response), {noreply, NewState} end; handle_info({abort, MsgId}, State) -> case get_from(MsgId, State) of undefined -> {noreply, State}; {_, NewState} -> {noreply, NewState} end. From hd2010@REDACTED Wed May 12 19:16:40 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 12 May 2010 19:16:40 +0200 Subject: scan to list Message-ID: <4BEAE278.70709@eonblast.com> Hello list, is there a better/shorter way to write this: <> = Stream, ColumnTypes = binary_to_list(ColumnTypeBinaries), Thanks a lot! Henning From igorrs@REDACTED Wed May 12 19:26:48 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 12 May 2010 14:26:48 -0300 Subject: [erlang-questions] how to clear a hot mnesia table In-Reply-To: References: Message-ID: Hi, Brian. It's been a long time since you asked that question. Did someone help you with that? I probably would do that using a "dirty cursor" to iterate through the table keys and delete them one by one. Does that create any problems for you? Table = your_table, Cursor = mnesia:activity(sync_dirty, fun() -> qlc:cursor(qlc:q([element(2, R) || R <- mnesia:table(Table)])) end). And then repeatedly: - Call qlc:next_answers(Cursor) to get a list with the next keys. - Call mnesia:activity(sync_transaction, fun mnesia:delete/1, [{Table, Key}]) for each returned key. You can also introduce some artificial delays to avoid interfering too much with the normal operation of your system. Igor. On Thu, Apr 1, 2010 at 9:12 PM, Brian Acton wrote: > Hi guys, > > I am running R13B04 SMP on FreeBSD 7.3 > > I have a very large table (i.e. 5.98M records covering 1.66GB) in Mnesia > that I need to clear out. Unfortunately, the transaction volume is > reasonably high and the servers are 24/7 servers. > > I'd like to clear out the table without experiencing any downtime. > > I know about mnesia:clear_table. However, I am concerned that since this > requires transactional semantics that mnesia will succeed on one node but > basically bring down the rest of the nodes as transactions pile up waiting > on some form of table lock. > > Is there a clean and simple way to empty out the table in a less risky > fashion ? > > --b From dougedmunds@REDACTED Wed May 12 19:55:19 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Wed, 12 May 2010 10:55:19 -0700 Subject: [Announcement] wx module tutorial Message-ID: <4BEAEB87.1010607@gmail.com> I've posted a tutorial on using the wx module at http://wxerlang.dougedmunds.com -- Doug Edmunds search tags for http://www.erlang.org/cgi-bin/swish.cgi [ gui window graphical interface ] From hd2010@REDACTED Wed May 12 19:56:14 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 12 May 2010 19:56:14 +0200 Subject: wise binary tails? Message-ID: <4BEAEBBE.1030405@eonblast.com> If I have code like this: traverse(<>) -> [ Bar | traverse(Rest) ]; traverse(<<>>) -> []. traverse(FooBin). - or - [ Bar || <> <= FooBin ]. is Erlang internally doing the wise thing of simply shifting a pointer into the low level buffer where FooBin is stored. And thus avoids copying the rest of the buffer again and again? Thanks! Henning From kiszl@REDACTED Wed May 12 19:58:20 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Wed, 12 May 2010 19:58:20 +0200 Subject: [erlang-questions] scan to list In-Reply-To: <4BEAE278.70709@eonblast.com> References: <4BEAE278.70709@eonblast.com> Message-ID: <4BEAEC3C.50803@tmit.bme.hu> On 5/12/2010 7:16 PM, Henning Diedrich wrote: > Hello list, > > is there a better/shorter way to write this: > > <> = Stream, > ColumnTypes = binary_to_list(ColumnTypeBinaries), > > Thanks a lot! > Henning > Well... are you sure you have to convert that binary to a list ? :o) From hd2010@REDACTED Wed May 12 20:05:09 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 12 May 2010 20:05:09 +0200 Subject: [erlang-questions] scan to list In-Reply-To: <4BEAEC3C.50803@tmit.bme.hu> References: <4BEAE278.70709@eonblast.com> <4BEAEC3C.50803@tmit.bme.hu> Message-ID: <4BEAEDD5.2040704@eonblast.com> :-) I would try to excuse that with 'for the sake of clarity.' Yes, these are numbers that happen to come in one byte integers and my be mixed with other numbers that are bigger than one byte. Zoltan Lajos Kis wrote: > On 5/12/2010 7:16 PM, Henning Diedrich wrote: >> Hello list, >> >> is there a better/shorter way to write this: >> >> <> = Stream, >> ColumnTypes = binary_to_list(ColumnTypeBinaries), >> >> Thanks a lot! >> Henning >> > Well... are you sure you have to convert that binary to a list ? :o) > From dmurray@REDACTED Wed May 12 17:07:17 2010 From: dmurray@REDACTED (David N Murray) Date: Wed, 12 May 2010 11:07:17 -0400 (EDT) Subject: [erlang-questions] TCP receive in gen_server In-Reply-To: References: Message-ID: On May 12, David N Murray scribed: > On May 12, David N Murray scribed: > > > I'm still seeing the same problem. No data from the socket is sent to > > handle_info unless I first send something to the socket, and then, only > > one packet/message from the server is delivered at a time, even though > > there are at least 3 messages pending: > > I figured it out. I had to call inet:setopts(Socket, [{active, once}]) > after the gen_tcp:send/2, too. > I spoke too soon. What's running fine on Linux doesn't work on Windows. Same problem. inet:setopts always returns ok. inet:getopts prior to a send returns {active, once} Does anyone have a suggestion of some source code that implements gen_server behavior, receiving from a socket via handle_info()? I'm not finding much in google. Thanks, Dave From caio.ariede@REDACTED Wed May 12 20:27:46 2010 From: caio.ariede@REDACTED (caio ariede) Date: Wed, 12 May 2010 15:27:46 -0300 Subject: Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: Message-ID: Someone found or have a walkthrough to workaround this problem? Caio Ariede http://caioariede.com/ On Tue, May 11, 2010 at 10:09 AM, caio ariede wrote: > Both platforms give me the same error, just alterning the text specifying > the platform ("x86_64-pc-linux-gnu") and the path, on the error below: > > 1> wx:new(). > > =ERROR REPORT==== 11-May-2010::10:01:54 === > WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: > /usr/lib/erlang/lib/wx-0.98.2/priv > ** exception error: {load_driver,"No driver found"} > in function wxe_server:start/0 > in call from wx:new/1 > > > Caio Ariede > http://caioariede.com/ > From hd2010@REDACTED Wed May 12 20:34:06 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 12 May 2010 20:34:06 +0200 Subject: immutable benefits Message-ID: <4BEAF49E.9060005@eonblast.com> Another hoping for magic. Is it possibly the case that Erlang will not copy anything from ColumnNamesBinary in the following example but simply have an internal representation of two pointers for each of the resulting 'Name' binaries, which pointers would point into ColumnNamesBinary, to the beginning of each respective individual name, and to its end? (Or using a length instead). ColumnNames = [ Name || <> <= ColumnNamesBinary ]. Just asking since I could imagine that this could be the case to save a lot of space and performance. I thought the immutability of variables may allow for this strategy. Although I would imagine that garbage collection may severely hampered by such a strategy. Thanks for letting us know, Henning From max.lapshin@REDACTED Wed May 12 20:50:39 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 May 2010 22:50:39 +0400 Subject: [erlang-questions] immutable benefits In-Reply-To: <4BEAF49E.9060005@eonblast.com> References: <4BEAF49E.9060005@eonblast.com> Message-ID: "Erlang performance" part of documentation tells us, that this code will create two subbinaries, that are pointing into refcounted large binary. For example, this way is excelent for mmap-reading of files, but it is not yet implemented. From hd2010@REDACTED Wed May 12 21:18:48 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 12 May 2010 21:18:48 +0200 Subject: [erlang-questions] immutable benefits In-Reply-To: References: <4BEAF49E.9060005@eonblast.com> Message-ID: <4BEAFF18.5010100@eonblast.com> Thanks for heads up, now reading ... http://www.erlang.org/doc/efficiency_guide/introduction.html - or - http://www.erlang.org/doc/pdf/otp-system-documentation.pdf pg 164 ... And what part is not yet implemented? Max Lapshin wrote: > "Erlang performance" part of documentation tells us, that this code > will create two subbinaries, that are pointing into refcounted large > binary. > > For example, this way is excelent for mmap-reading of files, but it is > not yet implemented. > > From aj@REDACTED Wed May 12 21:34:17 2010 From: aj@REDACTED (AJ Heller) Date: Wed, 12 May 2010 12:34:17 -0700 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: I would certainly appreciate a single reference on high-level distributed/parallel design patterns. I can find some of the patterns you list in a handful of books and papers, but not all of them in one place. Over the past year or so, I've searched for a compiled list like you're suggesting, and I haven't turned up much. So even if such a thing does exist, I'm probably not the only one who hasn't found it yet. While having implementations of these patterns would be great, I'd just like to add that if a list of patterns must be compiled to begin with, that list would be of great benefit to a broader audience than just the erlang community. I'm thinking in particular of a GoF-style of pattern analysis wherein you offer lists of common uses, strengths, and drawbacks for each pattern. A work like that on high-level parallel/distributed design patterns would be invaluable. @Vlad: While I empathize with you, I do think the distinctions are important. Taking myself as an example, I can still look at most of the GoF patterns and argue that they're each a special case of the Strategy Pattern (though I've learned to stop arguing about this). To people that work with these patterns on a daily basis, the nuances between the patterns are significant. So even though I don't yet grasp many of the nuances myself, I can appreciate that other people do. So long as there are significant differences between appropriate uses, strengths, and pitfalls for a pattern and its variation, I'd argue that the variation probably deserves its own entry in the list. Even if they all still look like divide and conquer to me :) . On Wed, May 12, 2010 at 3:35 AM, Vlad Dumitrescu wrote: > Hello Joe, > > On Wed, May 12, 2010 at 09:31, Joe Armstrong wrote: >> I'm interested in trying to classify and name a number of different >> parallelization strategies. > >> 1) Divide and conquer concurrency >> 2) Pipeline concurrency >> 3) Map reduce concurrency >> 4) Identical Job concurrency >> 5) Grid concurrency >> 6) Job queue concurrency > > IMHO, #1, #3 and #4 are the same thing, differently parametrized. Do > you think it is important to get them separate identities? > > Regarding #5, a more general description would involve a grid that is > not restricted to two dimensions. I think I remember seeing papers > about 3D grids. > > #6 can be seen as a generalization of #2, where in the pipeline the > "next" worker is hardcoded and the structure is linear. A pipeline may > or may not use buffering queues between stages. > > The way I see it, there are two main categories: > * dividing the data set into smaller chunks and processing them in > parallel (#1,3,4) > * dividing the work into specialized chunks and letting the data flow > to next chunk (#2,6) > > These two can also be combined in several ways, giving for example #5 > (usually each processor has its own chunk of input data, but data can > flow to neighboring processors). Or each processing unit in #1 could > involve a pipeline. > > It depends on the purpose of defining these categories whether a > shorter, more general list or a more detailed one is required. > > best regards, > Vlad > >> 1) Divide and conquer concurrency >> >> Divide the problem into K jobs which can be performed in parallel. >> Perform the jobs >> Recombine the results >> >> 2) Pipeline concurrency >> >> Divide the problem into K jobs that can be performed in a pipeline. >> The output of the first job must be the input of the second and so on. >> Run all the steps in parallel >> >> 3) Map reduce concurrency >> >> ? ?This is similar to 1). Only the recombination step involves >> merging results that >> ? have the same Key. >> >> 4) Identical Job concurrency >> >> ? ?Here the problem does not have to be split into K parts, we >> already have N identical >> ? ?jobs to do (identical means jobs that will consume similar resources) >> >> ? There is no recombination step. There are N results. >> >> 5) Grid concurrency >> >> ? ?Divide the problem into N*M identical jobs that can be solved on a >> NxM grid of processors. >> ? ?Each processor can only talk to its direct neighbors >> >> 6) Job queue concurrency >> >> ? ?Divide the problem into a set of named workers (who do jobs). Each >> worker has an input >> ? ?job queue. Workers consume jobs from their job queues and send the >> result to other workers. >> >> ? ?Note: ?[1] Paul Morrison calls this flow based programming > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From vladdu55@REDACTED Wed May 12 22:32:32 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 12 May 2010 22:32:32 +0200 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: Hi AJ, On Wed, May 12, 2010 at 21:34, AJ Heller wrote: > @Vlad: While I empathize with you, I do think the distinctions are > important. Taking myself as an example, I can still look at most of > the GoF patterns and argue that they're each a special case of the > Strategy Pattern (though I've learned to stop arguing about this). To > people that work with these patterns on a daily basis, the nuances > between the patterns are significant. So even though I don't yet grasp > many of the nuances myself, I can appreciate that other people do. So > long as there are significant differences between appropriate uses, > strengths, and pitfalls for a pattern and its variation, I'd argue > that the variation probably deserves its own entry in the list. Even > if they all still look like divide and conquer to me :) . I don't disagree. The only item that really isn't a separate one is #4 (identical jobs) which is identical to #1 - it only happens that the number of jobs is the same as the number of processing units that I have available here and now. On most other machines it would be classified as #1. best regards, Vlad From rvirding@REDACTED Thu May 13 00:40:05 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 13 May 2010 00:40:05 +0200 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: I quite agree with all Richards comments/critiques and would like to add one more: (7) It imposes a directory structure on packages which I think is undesirable. I do not want the language to force me to structure my apps in a special way. Currently all that OTP assumes is that the source code for a module is in a file called .erl and the BEAM code is in a file called .beam. This, I think, is quite enough. It is actually not too difficult to get around the second one if you are prepared to load code yourself. Otherwise I can add '_' in my module names myself. -1 Robert On 12 May 2010 05:52, Richard O'Keefe wrote: > > On May 11, 2010, at 9:53 PM, Max Lapshin wrote: > >> Hi, I can't understand why is not recommended now to use packages of >> modules. > > There are several important things here. > > (1) Erlang allows dots in atoms, always has. ?foo.bar.ugh > ? ?and foo@REDACTED@ugh and foo_bar_ugh are just plain atoms. > ? ?They are the _same_ atoms as 'foo.bar.ugh', 'foo@REDACTED@ugh', > ? ?and 'foo_bar_ugh'. > > ? ?This does _not_ apply to Java, where foo_bar is one > ? ?identifier but foo.bar is not. > > (2) Historically, the module system treated all atoms alike. > ? ?There was no connection between foo.bar and bar any more > ? ?than there was any connection between foo@REDACTED and bar or > ? ?between foo_bar and bar. ?In the same way, Java sees no > ? ?connection between a class called Foo_Bar and a class > ? ?called Bar. > > ? ?One of the fundamental concepts in computer programming > ? ?is alpha-conversion (the idea that names are *arbitrary* > ? ?labels). ?It is generally a bad idea to violate this. > > (3) Formerly, in -module(fred), a call jill:bucket(56) was a > ? ?call to the module 'jill'. ?Change the -module directive > ? ?to -module(demo.fred) and that call now refers to the > ? ?module 'demo.jill'. ?This is fine when you want it to > ? ?happen, but if your code referred to existing OTP modules > ? ?such as 'lists', well hello demo.lists breakage. ?You _can_ > ? ?refer to top level modules using ''.lists or .lists, but > ? ?that's an incompatible change. > > (4) It's really not clear how all of this works *dynamically*. > ? ?Before dotted module names got special treatment, > ? ? ? ?jill:bucket(56), ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?%A > ? ? ? ?M = jill, M:bucket(56), ? ? ? ? ? ? ? ? ? ? ? ? %B > ? ? ? ?apply(jill, bucket, [56]), ? ? ? ? ? ? ? ? ? ? ?%C > ? ? ? ?'my.module':'my.apply'(jill, bucket, [56]) ? ? ?%D > ? ?were all equivalent, where > ? ? ? ?-module('my.module'). > ? ? ? ?-export(['my.apply'/3]). > ? ? ? ?my.apply(M, F, A) -> apply(M, F, A). > ? ?The abbreviation kluge obviously works for %A. ?It could > ? ?be made to work for %B by using different (slower!) code > ? ?like M = jill, (fix_mod(M, 'my.')):bucket(56), where > ? ? ? ?fix_mod(M, C) -> case has_dots(M) > ? ? ? ? ? ? ? ? ? ? ? ? ? of true ?-> M > ? ? ? ? ? ? ? ? ? ? ? ? ? ?; false -> atom_append(C, M) > ? ? ? ? ? ? ? ? ? ? ? ? end. > ? ?It's easy to make %C work when the first argument is a > ? ?constant, like %A, and when the first argument is not a > ? ?constant, the same technique used for %B will work. > > ? ?But how do you make %D work? > > (5) There are no actual packages. ?There are directories where > ? ?modules with the same prefix will be sought, but it is not > ? ?the case that all files with the same prefix must be in the > ? ?same directory. ?A "package" is nothing but a bunch of modules > ? ?with a common prefix to their names; there are no package > ? ?properties. > > (6) Package names are back to front. ?This has been explained repeatedly > ? ?in this mailing list. ?They are like absolute file names, not like > ? ?relative ones. ?If I need the same package in two places in the > ? ?package hierarchy, I can't do it. ?I can't move a package up or > ? ?down in the package hierarchy without surgery on every file. ?I > ? ?cannot make a *component* that my customers can place anywhere > ? ?they want in the package hierarchy. ?And if I can't do *that*, > ? ?what the heck is the package system *for*? > > So: > ?- dotted module names was not a new thing > ?- mapping dotted names to the file system as described in > ? ?http://www.erlang.se/publications/packages.html > ? ?obviously has its uses, but this could be done with any > ? ?character, including @, or indeed by other means entirely > ? ?(like Eiffel's LACE) > ?- the scheme leads to increased work when adopting dotted names > ? ?and makes renaming packages difficult and fragile > ?- it does not seem that dynamic behaviour can match the static > ? ?abbreviation facility without increased costs (when known > ? ?functions/forms are involved) or at all (when user defined > ? ?functions are involved) > >> What is bad in >> using dotted names? What is the future of this feature: it will be >> developed or it will be deprecated? > > It should be taken out and shot. > > *Some* sort of module connection/configuration facility is needed, > but I suggest as a design criterion that it should be possible to > move an entire "flotilla" of modules in the package name space > by changing *ONE* line in some file. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bernie@REDACTED Thu May 13 01:21:22 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 13 May 2010 09:21:22 +1000 Subject: [erlang-questions] Timeouted gen_server:call In-Reply-To: <4BEABFB9.5060103@nm.ru> References: <4BEABFB9.5060103@nm.ru> Message-ID: <4BEB37F2.9030203@m5net.com> On 13/05/10 00:48, Dmitry Belyaev wrote: > I have a gen_server and many clients. Clients send request to the server > with gen_server:call(Server, Msg, Timeout) and when timeout occurs they > ask server to abort appropriate reply. But server may reply right before > it receives abort message and I may get trash in client's mailbox. > > I'm not sure that client can throw away this trash itself. So it is > possible for client to fill his message queue with lots of such messages. > Why can't the client clean up its queue itself? A general rule of thumb, in my experience at least, is that processes should flush their message queue of any "unexpected" messages - if not all the time, at least regularly. Not because they /should/ expect to get them, but because you don't want them to misbehave if they do (you're right to be concerned about a growing queue - as several people on this list, myself included, have discovered it can be "bad" when a queue gets too big). In the case of a gen_server and similar, this is easy - just add a catch-all handle_info(_, State) -> {noreply, State} In the case of non "gen style" code, just make sure it regularly reaches and loops on a 'receive' block which has a clause which matches everything you're not interested in and discards it. Cheers, Bernard From essiene@REDACTED Thu May 13 01:29:50 2010 From: essiene@REDACTED (Essien Essien) Date: Thu, 13 May 2010 00:29:50 +0100 Subject: [erlang-questions] gen_server cleanup handling In-Reply-To: <4BE9FA93.4080709@m5net.com> References: <4BE8B9BB.7070809@m5net.com> <4BE8FFFB.4080406@erlang-solutions.com> <4BE9FA93.4080709@m5net.com> Message-ID: Hi Bernard, 2010/5/12 Bernard Duggan : > Hi Mazen, > ? ?Our setup kind of has the opposite requirement - if either the worker or > the gen_server that spawned it crash, both should crash out (sorry, that > probably wasn't very clear in my initial post). ?That's why initially a > straight spawn_link to create the worker looked so attractive - errors > propagate between the gen_server and worker, both are cleaned up, and the > gen_server's supervisor just restarts the gen_server (which is what we > want). ?That, though, would require us to handle the EXIT message from the > worker in the gen_server if the latter turned on trap_exit (which it seems > like it needs to). ?It's not a huge problem, just seems like something we > should be able to avoid. I'm currently facing a similar situation (not exactly alike, but alike enough) and got tripped up by the need to trap_exit if you want to run terminate/2 on shutdown. What I'm starting to realize is that linking should as much as possible, ONLY be used withing the OTP supervision tree context... and monitors should be used for everything else. I've not seen this written anywhere, but I've come to discover that there are OTP ways to work around the scenario you're describing. Managing it any other way will break in some way within OTP. Now i'm experimenting with depending on the various supervisor restart strategies, children restart types and restart frequencies, monitors, etc. - You want a gen_server to spawn a worker, and both should be restarted if any of the other goes down... What to do is this (forgive the ascii art!): [root_sup] % root_sup should use a one_for_all restart strategy | [gen_server, worker_sup] % worker_sup should be simple_one_for_one, MaxR = 0, MaxT=10 | [worker(s)] % this should be a permanent child. To complete the above, establish a monitor (erlang:monitor/2) b/w gen_server and worker pid after the supervisor:start_child(worker_sup, []) call. The above tree will work as you want, though it introduces root_sup and worker_sup. - If gen_server crashes, worker_sup will also crash and restart, thus taking down and restart worker(s)? - Since the gen_server is monitoring the worker, it will recieve the 'DOWN' message when the worker goes down and you can then {stop, Reason} from the gen_server. Also, the worker restart frequency is zero times in 10 seconds, so it will not be restarted by its supervisor, untill the gen_server has died also, taken them all down and they're all restarted by the root_sup. I have tried it out, and it works nicely too. The question i have for you though, is: Does every gen_server just spawn a single worker process or can it spawn multiple workers? If multiple, what happens to the rest if one dies? (I'm assuming the answer to this is one gen_server, one worker) Hope that helps. cheers, Essien > > On 11/05/10 16:58, Mazen Harake wrote: >> >> Spawn your worker processes in a simple_one_for_one supervisor which is >> under your supervisor. This will propagate the exit to the worker >> processes. Use the gen_server under the first Sup to start a worker >> child in the simple_one_for_one Sup. >> >> In other words; Garrett seems to have gotten it right. >> >> This also eliminates point 3 because the crash will not propagate >> through the gen_server. >> >> I'm assuming from your setup that the gen_server doesn't care if the >> worker crashed or not before it finished, otherwise you might as well >> just trap exits and handle all the exit messages anyway without a >> supervisor. >> >> Hope this makes sense; it is morning here and my coffee cup is still >> full (didn't have a chance to drink it yet :)) >> >> Good luck >> >> /Mazen >> >> On 11/05/2010 04:58, Bernard Duggan wrote: >> >>> >>> Hi list, >>> ? ? This is something of a followup to my previous question about >>> supervision trees. ?Basically we have a gen_server that, in the case >>> of a standard app shutdown, needs to do some cleanup. ?A standard app >>> shutdown seems to take the form of supervisors sending 'shutdown' exit >>> messages to their children. ?Fine so far. ?The catch is that in order >>> for the terminate() handler to be invoked on our server, we have to >>> turn on trap_exit in every gen_server that needs to do cleanup. ?This >>> seems non-ideal for a few reasons: >>> >>> * Elsewhere in the docs, we're cautioned against using trap_exit >>> except when unavoidable (though I'm happy to accept it if this is one >>> such case). >>> >>> * It means that we don't get automatic propagation of crashes from any >>> worker processes we might spawn. ?Instead we have to write a >>> handle_info({'EXIT'...) in every gen_server that might ever link to >>> another process to ensure those crashes are propagated properly. ?This >>> seems like it could be solved by instead spawning any worker processes >>> as a child of our supervisor (which is what Garrett suggested we >>> should do anyway) - if that's a good reason to set up things that way, >>> I'm likewise happy to accept it and rearrange our code accordingly. >>> >>> * Most concerning, though, is the possibility of some library call >>> creating a link to a temporary worker process (or any process for that >>> matter) whose crash should propagate through us - in this case we'd >>> still have to have the handle_info({'EXIT'...) setup as a catchall >>> which seems like a fiddly, repetitive bit of code we'd rather avoid if >>> possible. >>> >>> So what's the thinking about this? ?Am I missing something obvious? >>> Should I just turn on trap_exit willy-nilly wherever I need shutdown >>> cleanup? ?Should I just suck it up and write the 'EXIT' message >>> handlers in all such gen_servers? >>> >>> Cheers, >>> >>> Bernard >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become >> ERLANG SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hd2010@REDACTED Thu May 13 01:37:01 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 13 May 2010 01:37:01 +0200 Subject: [erlang-questions] immutable benefits In-Reply-To: <4BEAFF18.5010100@eonblast.com> References: <4BEAF49E.9060005@eonblast.com> <4BEAFF18.5010100@eonblast.com> Message-ID: <4BEB3B9D.10806@eonblast.com> So with the benefits being in fact as big as I had hoped for, I am wondering if I am getting this right: binary scans do indeed cascade down several levels and may all end up pointing to the same source chunk of memory? Plowing through the manual, what I am understanding is this (great news): ** *With Erlang you get for free that you (often) don't duplicate any contents at all when parsing binaries. *Although you may end up having a lot of variables. But they often all share the same contents buffer. You do have the time consumption of the parsing of course. But all that happens is that pointers into the binary are created. Nothing is copied. (See pg. 168+ Erlang/OTP System Documentation) Concretely, it does not copy the names /*Name*/ out of the source binary _*Bin*_ but only sets pointers into _*Bin*_ to populate the /*Name*/s. erl_table(<>=_*Bin*_) -> ... <> = _*Stream*_, ... % List comprehension from a binary generator: Names = [ Name || <> <= _*NamesBinary*_ ]. If I understand correctly, this leaves me with the individual /*Name */variables internally being but pointers into the original binary buffer of _*Bin*_* (*first line)? Thanks, Henning Henning Diedrich wrote: > Thanks for heads up, now reading ... > > http://www.erlang.org/doc/efficiency_guide/introduction.html > - or - http://www.erlang.org/doc/pdf/otp-system-documentation.pdf pg > 164 ... > > And what part is not yet implemented? > > Max Lapshin wrote: >> "Erlang performance" part of documentation tells us, that this code >> will create two subbinaries, that are pointing into refcounted large >> binary. >> >> For example, this way is excelent for mmap-reading of files, but it is >> not yet implemented. >> >> > > From raould@REDACTED Thu May 13 01:44:21 2010 From: raould@REDACTED (Raoul Duke) Date: Wed, 12 May 2010 16:44:21 -0700 Subject: [erlang-questions] immutable benefits In-Reply-To: <4BEB3B9D.10806@eonblast.com> References: <4BEAF49E.9060005@eonblast.com> <4BEAFF18.5010100@eonblast.com> <4BEB3B9D.10806@eonblast.com> Message-ID: On Wed, May 12, 2010 at 4:37 PM, Henning Diedrich wrote: > So with the benefits being in fact as big as I had hoped for, I am wondering > if I am getting this right: binary scans do indeed cascade down several > levels and may all end up pointing to the same source chunk of memory? as an aside, i wonder if one must be careful like with Java Strings: keeping a reference to a small portion of the whole thing prevents the whole thing from being garbage collected. From hd2010@REDACTED Thu May 13 02:10:03 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 13 May 2010 02:10:03 +0200 Subject: [erlang-questions] immutable benefits In-Reply-To: References: <4BEAF49E.9060005@eonblast.com> <4BEAFF18.5010100@eonblast.com> <4BEB3B9D.10806@eonblast.com> Message-ID: <4BEB435B.80300@eonblast.com> That's addressed on pg 172. I am not sure if it is the whole answer, since "shrinking" is used in a different context here: "If a process simply keeps binaries (either in "loop data" or in the process dictionary), the garbage collector may eventually shrink the binaries. If only one such binary is kept, it will not be shrunk." Shrinking, in the rest of the paragraph only addresses the deletion of 'reserve space' that is allocated in advance to allow for faster appending. Raoul Duke wrote: > On Wed, May 12, 2010 at 4:37 PM, Henning Diedrich wrote: > >> So with the benefits being in fact as big as I had hoped for, I am wondering >> if I am getting this right: binary scans do indeed cascade down several >> levels and may all end up pointing to the same source chunk of memory? >> > > as an aside, i wonder if one must be careful like with Java Strings: > keeping a reference to a small portion of the whole thing prevents the > whole thing from being garbage collected. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From bernie@REDACTED Thu May 13 02:29:01 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 13 May 2010 10:29:01 +1000 Subject: [erlang-questions] gen_server cleanup handling In-Reply-To: References: <4BE8B9BB.7070809@m5net.com> <4BE8FFFB.4080406@erlang-solutions.com> <4BE9FA93.4080709@m5net.com> Message-ID: <4BEB47CD.6080300@m5net.com> Hi Essien, On 13/05/10 09:29, Essien Essien wrote: > - You want a gen_server to spawn a worker, and both should be > restarted if any of the other goes down... What to do is this (forgive > the ascii art!): > > [root_sup] % root_sup should use a one_for_all restart strategy > | > [gen_server, worker_sup] % worker_sup should be simple_one_for_one, > MaxR = 0, MaxT=10 > | > [worker(s)] % this should be a permanent child. > > To complete the above, establish a monitor (erlang:monitor/2) b/w > gen_server and worker pid after the supervisor:start_child(worker_sup, > []) call. > > The above tree will work as you want, though it introduces root_sup > and worker_sup. > > - If gen_server crashes, worker_sup will also crash and restart, thus > taking down and restart worker(s)? > > - Since the gen_server is monitoring the worker, it will recieve the > 'DOWN' message when the worker goes down and you can then {stop, > Reason} from the gen_server. Also, the worker restart frequency is > zero times in 10 seconds, so it will not be restarted by its > supervisor, untill the gen_server has died also, taken them all down > and they're all restarted by the root_sup. > Yeah, that arrangement was one I considered, I'm just not sure that it's better than just doing a straight spawn_link, and catching and propagating the non-normal 'EXIT' message from the worker. If you have to catch and propagate the 'DOWN', the only difference in terms of code size is the extra supervisor and the explicit monitoring call which both count against that plan. (Not, of course, that code size should be the sole determining factor, but trying to avoid unnecessary code was what got me onto this train of thought in the first place). I do make use of monitoring elsewhere, though, and I think you're quite right that it's underused at the expense of linking. > I have tried it out, and it works nicely too. > > The question i have for you though, is: > > Does every gen_server just spawn a single worker process or can it > spawn multiple workers? If multiple, what happens to the rest if one > dies? (I'm assuming the answer to this is one gen_server, one worker) > You assume correctly. I have a gen_server that needs to be quite responsive. Occasionally it also kicks off a single longer-running operation which can be done in parallel, but is only transient. You know, it's just occurred to me that the "easy" and (probably) "right" solution for this particular case is to have the worker as "just another gen_server" under a one_for_all supervisor with the original gen_server. It can then sit idle quite happily until required - I don't know why I had it fixed in my head that it had to be spawned only when it was needed and terminated when it was done. Duh. Thanks for your thoughtful response :) Cheers, Bernard From luismarianoguerra@REDACTED Thu May 13 02:32:51 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Wed, 12 May 2010 21:32:51 -0300 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 12, 2010 at 5:32 PM, Vlad Dumitrescu wrote: > Hi AJ, > > On Wed, May 12, 2010 at 21:34, AJ Heller wrote: >> @Vlad: While I empathize with you, I do think the distinctions are >> important. Taking myself as an example, I can still look at most of >> the GoF patterns and argue that they're each a special case of the >> Strategy Pattern (though I've learned to stop arguing about this). To >> people that work with these patterns on a daily basis, the nuances >> between the patterns are significant. So even though I don't yet grasp >> many of the nuances myself, I can appreciate that other people do. So >> long as there are significant differences between appropriate uses, >> strengths, and pitfalls for a pattern and its variation, I'd argue >> that the variation probably deserves its own entry in the list. Even >> if they all still look like divide and conquer to me :) . > > I don't disagree. The only item that really isn't a separate one is #4 > (identical jobs) which is identical to #1 - it only happens that the > number of jobs is the same as the number of processing units that I > have available here and now. On most other machines it would be > classified as #1. I think that the first is a reference to a job where there is only one result and the job is divided to handle different parts and then recombine them. for example you are rendering one frame with ray tracing and divide the rendering in N horizontal parts and then recombine the result to get the full frame. the number 4 is more like having to render 24 frames to make 1 second of an animation, you divide the rendering, but you get 24 different results, you don't have to combine them later* * except if you want to make a video, maybe this is not a good example after all :D From kyleqian@REDACTED Thu May 13 07:09:30 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Thu, 13 May 2010 13:09:30 +0800 Subject: Questions about PID of remote process Message-ID: Hi, I have two questions about pid of remote process: 1. How do I know if the remote process identified by one PID is alive? I know I can send a message to it, and receive response with timeout. But I hope there is some way more conveniently. 2. Can the PID be reused in one node? If a process identified by one PID died, then this PID can be used to identify other process created later? Thanks! From bernie@REDACTED Thu May 13 07:17:20 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 13 May 2010 15:17:20 +1000 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: References: Message-ID: <4BEB8B60.7000707@m5net.com> Hi ???, On 13/05/10 15:09, ??? wrote: > 1. How do I know if the remot process identified by one PID is alive? I > know I can send a message to it, and receive response with timeout. But I > hope there is some way more conveniently. > One option (the one we use) is erlang:monitor(process, Pid). If he Pid dies or the node goes down, or is down when you make the monitor() call you'll get a message notifying you of the fact. > 2. Can the PID be reused in one node? If a process identified by one PID > died, then this PID can be used to identify other process created later? > Not as far as I know - the fact that you can call node(Pid) to get the node on which a PID resides, even after that PID has terminated, implies that the PID embeds (or at least links to) unique node information. Cheers, Bernard From ulf.wiger@REDACTED Thu May 13 07:50:29 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 May 2010 07:50:29 +0200 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: References: Message-ID: <4BEB9325.4090806@erlang-solutions.com> On 05/13/2010 07:09 AM, ??? wrote: > Hi, I have two questions about pid of remote process: > > 1. How do I know if the remote process identified by one PID is alive? I > know I can send a message to it, and receive response with timeout. But I > hope there is some way more conveniently. If you monitor it, you will be notified when it dies - or immediately if it is already dead. While you /could/ call rpc:call(node(Pid),erlang,is_process_alive,[P]), this will only sample the status of the Pid, and you know nothing more than that it was (possibly) alive at the time of the call. > 2. Can the PID be reused in one node? If a process identified by one PID > died, then this PID can be used to identify other process created later? Not if you monitor the process. The monitor is a known reference to the Pid, so it will not be reused. Otherwise, the garbage collector has no knowledge of references to the Pid kept on disk or on other nodes. BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From bernie@REDACTED Thu May 13 08:16:13 2010 From: bernie@REDACTED (Bernard Duggan) Date: Thu, 13 May 2010 16:16:13 +1000 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <4BEB9325.4090806@erlang-solutions.com> References: <4BEB9325.4090806@erlang-solutions.com> Message-ID: <4BEB992D.2030301@m5net.com> On 13/05/10 15:50, Ulf Wiger wrote: > >> 2. Can the PID be reused in one node? If a process identified by one PID >> died, then this PID can be used to identify other process created later? >> > Not if you monitor the process. The monitor is a known reference to the > Pid, so it will not be reused. Otherwise, the garbage collector has no > knowledge of references to the Pid kept on disk or on other nodes. > Ah, right - thanks for clarifying that, Ulf. Does the monitor continue to remain a "known reference" after it sends the {'DOWN'...} message about the process? If so, what, if any, condition /does/ release a monitored PID for garbage collection? Cheers, Bernard From mazen.harake@REDACTED Thu May 13 08:54:21 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 13 May 2010 09:54:21 +0300 Subject: [erlang-questions] Timeouted gen_server:call In-Reply-To: <4BEABFB9.5060103@nm.ru> References: <4BEABFB9.5060103@nm.ru> Message-ID: <4BEBA21D.5030604@erlang-solutions.com> Hi Dmitry, If the request can become outdated (I.e. the server should ignore it) then perhaps you can send the expiry along with the message. Example: do_call(Server, MsgId, Msg, Timeout) -> try gen_server:call(Server, {process, MsgId, Msg, calc_expiry_time(Timeout)}, Timeout); catch exit:{timeout, _} -> %% clean up here... ok end. implement calc_expirty_time and in the server you simply check if the message is old and if so throw it away. Just a thought, /Mazen On 12/05/2010 17:48, Dmitry Belyaev wrote: > I have a gen_server and many clients. Clients send request to the > server with gen_server:call(Server, Msg, Timeout) and when timeout > occurs they ask server to abort appropriate reply. But server may > reply right before it receives abort message and I may get trash in > client's mailbox. > > I'm not sure that client can throw away this trash itself. So it is > possible for client to fill his message queue with lots of such messages. > > I may not let client crash. Neither I don't want to spawn new process > for every such request. > > Looking through gen module I found that wait_resp*/3 could tell me > Mref with exit({timeout, Mref}). Then I could use gen:call() and catch > that Mref. But it currently doesn't offer such information. > > > So, I'd like to know how more experienced programmers workaround this > problem. > > > Code like this: > > -export([do_call/4, handle_call/3, handle_info/2]). > > do_call(Server, MsgId, Msg, Timeout) -> > try gen_server:call(Server, {process, MsgId, Msg}, Timeout) of > Response -> > Response > catch > exit:{timeout, Reason} -> > %% this is a place when server can reply us > Server ! {abort, MsgId}, > %% here I'd like to cleanup message queue but I don't know Mref > error > end. > > > handle_call({process, MsgId, Msg}, From, State) -> > start_processing(MsgId, Msg), > {noreply, State#state{ids=[{MsgId, From} | State#state.ids]}}. > > > handle_info({ready, MsgId, Response}, State) -> > case get_from(MsgId, State) of > undefined -> > {noreply, State}; > {From, NewState} -> > gen_server:reply(From, Response), > {noreply, NewState} > end; > > handle_info({abort, MsgId}, State) -> > case get_from(MsgId, State) of > undefined -> > {noreply, State}; > {_, NewState} -> > {noreply, NewState} > end. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mazen.harake@REDACTED Thu May 13 08:57:24 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 13 May 2010 09:57:24 +0300 Subject: [erlang-questions] [Announcement] wx module tutorial In-Reply-To: <4BEAEB87.1010607@gmail.com> References: <4BEAEB87.1010607@gmail.com> Message-ID: <4BEBA2D4.7070001@erlang-solutions.com> Very cool! Just 10 months too late :P Which this would've been around when I was developing my stuff. Really nice job thought... I'll look at it closer when I get off work :) /M On 12/05/2010 20:55, Doug Edmunds (gmail) wrote: > I've posted a tutorial on using the wx module > at http://wxerlang.dougedmunds.com > > -- Doug Edmunds > > search tags for > http://www.erlang.org/cgi-bin/swish.cgi > [ gui window graphical interface ] > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From nicolas.charpentier@REDACTED Thu May 13 09:54:54 2010 From: nicolas.charpentier@REDACTED (Nicolas Charpentier) Date: Thu, 13 May 2010 08:54:54 +0100 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: References: Message-ID: <36D9861C-8D89-446A-9AEC-5D943A1E2ED2@erlang-solutions.com> Hi, On 13 May 2010, at 06:09, ??? wrote: > Hi, I have two questions about pid of remote process: > > 2. Can the PID be reused in one node? If a process identified by one PID > died, then this PID can be used to identify other process created later? > PIDs are unique for all the life of one node, but if the node is restarted 'almost' the same pid will be used. The difference will be the node incarnation number. This value is not visible when you print the value of PID. So if you store a PID in a database with pid_to_list(PID) and retrieve it later with list_to_pid(List), you may send a message to the wrong process if the node on which the process was running crashed. Regards, Nicolas Charpentier --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From ulf.wiger@REDACTED Thu May 13 09:56:17 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 May 2010 09:56:17 +0200 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <4BEB992D.2030301@m5net.com> References: <4BEB9325.4090806@erlang-solutions.com> <4BEB992D.2030301@m5net.com> Message-ID: <4BEBB0A1.7030105@erlang-solutions.com> On 05/13/2010 08:16 AM, Bernard Duggan wrote: > On 13/05/10 15:50, Ulf Wiger wrote: >> >>> 2. Can the PID be reused in one node? If a process identified by one PID >>> died, then this PID can be used to identify other process created later? >>> >> Not if you monitor the process. The monitor is a known reference to the >> Pid, so it will not be reused. Otherwise, the garbage collector has no >> knowledge of references to the Pid kept on disk or on other nodes. >> > Ah, right - thanks for clarifying that, Ulf. Does the monitor continue > to remain a "known reference" after it sends the {'DOWN'...} message > about the process? If so, what, if any, condition /does/ release a > monitored PID for garbage collection? I will have to venture an educated guess here, and Bjorn can slap my fingers if I get it wrong... If A monitors B, the monitor will automatically go away if: - A dies - B dies (and 'DOWN' message is delivered to A) - A calls demonitor for the given monitor instance - If A and B are on different nodes, node(A) is disconnected from node(B). This doesn't necessarily mean that A died. Note that A can have several monitor instances towards B. The above will hold individually for each instance. BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rumata-estor@REDACTED Thu May 13 10:10:46 2010 From: rumata-estor@REDACTED (Dmitry Belyaev) Date: Thu, 13 May 2010 12:10:46 +0400 Subject: [erlang-questions] Timeouted gen_server:call In-Reply-To: <4BEBA21D.5030604@erlang-solutions.com> References: <4BEABFB9.5060103@nm.ru> <4BEBA21D.5030604@erlang-solutions.com> Message-ID: <4BEBB406.6000105@nm.ru> This won't help because server may send the message right before timeout happens (say 1ms). And client may receive the message right after the timeout happens (1ms after). Mazen Harake wrote: > > Hi Dmitry, > > If the request can become outdated (I.e. the server should ignore it) > then perhaps you can send the expiry along with the message. > > Example: > > do_call(Server, MsgId, Msg, Timeout) -> > try > gen_server:call(Server, {process, MsgId, Msg, > calc_expiry_time(Timeout)}, Timeout); > catch > exit:{timeout, _} -> > %% clean up here... > ok > end. > > implement calc_expirty_time and in the server you simply check if the > message is old and if so throw it away. > > Just a thought, > > /Mazen > > On 12/05/2010 17:48, Dmitry Belyaev wrote: >> I have a gen_server and many clients. Clients send request to the >> server with gen_server:call(Server, Msg, Timeout) and when timeout >> occurs they ask server to abort appropriate reply. But server may >> reply right before it receives abort message and I may get trash in >> client's mailbox. >> >> I'm not sure that client can throw away this trash itself. So it is >> possible for client to fill his message queue with lots of such >> messages. >> >> I may not let client crash. Neither I don't want to spawn new process >> for every such request. >> >> Looking through gen module I found that wait_resp*/3 could tell me >> Mref with exit({timeout, Mref}). Then I could use gen:call() and >> catch that Mref. But it currently doesn't offer such information. >> >> >> So, I'd like to know how more experienced programmers workaround this >> problem. >> >> >> Code like this: >> >> -export([do_call/4, handle_call/3, handle_info/2]). >> >> do_call(Server, MsgId, Msg, Timeout) -> >> try gen_server:call(Server, {process, MsgId, Msg}, Timeout) of >> Response -> >> Response >> catch >> exit:{timeout, Reason} -> >> %% this is a place when server can reply us >> Server ! {abort, MsgId}, >> %% here I'd like to cleanup message queue but I don't know Mref >> error >> end. >> >> >> handle_call({process, MsgId, Msg}, From, State) -> >> start_processing(MsgId, Msg), >> {noreply, State#state{ids=[{MsgId, From} | State#state.ids]}}. >> >> >> handle_info({ready, MsgId, Response}, State) -> >> case get_from(MsgId, State) of >> undefined -> >> {noreply, State}; >> {From, NewState} -> >> gen_server:reply(From, Response), >> {noreply, NewState} >> end; >> >> handle_info({abort, MsgId}, State) -> >> case get_from(MsgId, State) of >> undefined -> >> {noreply, State}; >> {_, NewState} -> >> {noreply, NewState} >> end. >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From b.nicholson@REDACTED Thu May 13 08:26:46 2010 From: b.nicholson@REDACTED (B. Nicholson) Date: Thu, 13 May 2010 01:26:46 -0500 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <4BEB8B60.7000707@m5net.com> References: <4BEB8B60.7000707@m5net.com> Message-ID: <4BEB9BA6.3050507@niceng.com> I thought that the PID of a process was made up of three pieces. . The low bits and the high bits combine to make up a 28 bit number. So, I think it is possible to create the same process id by creating 268435456 processes. I wrote a quick program to test this. -module(proctest). -export([loop/1]). loop(N) -> testspawn(N), loop(N+1). testspawn(N) -> Pid = spawn(fun() -> donothing() end), PidStr = pid_to_list(Pid), case PidStr of "<0.1000.0>" -> io:format("Process ~p found @ ~p (iteration ~p)~n",[Pid,calendar:local_time(),N]); _ -> ok end. donothing() -> _A = 1 + 2. The output follows: Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> c(proctest). {ok,proctest} 2> proctest:loop(). Process <0.1000.0> found @ {{2010,5,13},{1,14,45}} (iteration 959) Process <0.1000.0> found @ {{2010,5,13},{1,24,5}} (iteration 268223423) I think you have to account for 'duplicate' local pids if your Erlang node is up for a considerable amount of time or goes through large numbers of processes for some reason. Hope this helps. Barry Nicholson On 05/13/2010 12:17 AM, Bernard Duggan wrote: > Hi ???, > > On 13/05/10 15:09, ??? wrote: > >> 1. How do I know if the remot process identified by one PID is alive? I >> know I can send a message to it, and receive response with timeout. But I >> hope there is some way more conveniently. >> >> > One option (the one we use) is erlang:monitor(process, Pid). If he Pid > dies or the node goes down, or is down when you make the monitor() call > you'll get a message notifying you of the fact. > >> 2. Can the PID be reused in one node? If a process identified by one PID >> died, then this PID can be used to identify other process created later? >> >> > Not as far as I know - the fact that you can call node(Pid) to get the > node on which a PID resides, even after that PID has terminated, implies > that the PID embeds (or at least links to) unique node information. > > Cheers, > > Bernard > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From ulf.wiger@REDACTED Thu May 13 13:26:15 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 May 2010 13:26:15 +0200 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <4BEB9BA6.3050507@niceng.com> References: <4BEB8B60.7000707@m5net.com> <4BEB9BA6.3050507@niceng.com> Message-ID: <4BEBE1D7.3050000@erlang-solutions.com> Your test program does not keep a reference to the Pid, so each process you create will perform its very short job, then die, making that Pid available for reuse. If you would keep the pids in e.g. a list, there would be no duplication. BR, Ulf W On 05/13/2010 08:26 AM, B. Nicholson wrote: > I thought that the PID of a process was made up of three pieces. > > . > > The low bits and the high bits combine to make up a 28 bit number. So, I > think it is possible to create the same process id by creating 268435456 > processes. I wrote a quick program to test this. > > -module(proctest). > -export([loop/1]). > > loop(N) -> > testspawn(N), > loop(N+1). > > testspawn(N) -> > Pid = spawn(fun() -> donothing() end), > PidStr = pid_to_list(Pid), > case PidStr of > "<0.1000.0>" -> io:format("Process ~p found @ ~p (iteration > ~p)~n",[Pid,calendar:local_time(),N]); > _ -> ok > end. > > donothing() -> > _A = 1 + 2. > > The output follows: > > Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:8:8] [rq:8] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.4 (abort with ^G) > 1> c(proctest). > {ok,proctest} > 2> proctest:loop(). > Process <0.1000.0> found @ {{2010,5,13},{1,14,45}} (iteration 959) > Process <0.1000.0> found @ {{2010,5,13},{1,24,5}} (iteration 268223423) > > I think you have to account for 'duplicate' local pids if your Erlang > node is up for a considerable amount of time or goes > through large numbers of processes for some reason. > > Hope this helps. > > Barry Nicholson > > On 05/13/2010 12:17 AM, Bernard Duggan wrote: >> Hi ???, >> >> On 13/05/10 15:09, ??? wrote: >> >>> 1. How do I know if the remot process identified by one PID is alive? I >>> know I can send a message to it, and receive response with timeout. But I >>> hope there is some way more conveniently. >>> >>> >> One option (the one we use) is erlang:monitor(process, Pid). If he Pid >> dies or the node goes down, or is down when you make the monitor() call >> you'll get a message notifying you of the fact. >> >>> 2. Can the PID be reused in one node? If a process identified by one PID >>> died, then this PID can be used to identify other process created later? >>> >>> >> Not as far as I know - the fact that you can call node(Pid) to get the >> node on which a PID resides, even after that PID has terminated, implies >> that the PID embeds (or at least links to) unique node information. >> >> Cheers, >> >> Bernard >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From ok@REDACTED Thu May 13 13:26:47 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 13 May 2010 23:26:47 +1200 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <4BEB9325.4090806@erlang-solutions.com> References: <4BEB9325.4090806@erlang-solutions.com> Message-ID: <9A99A5EF-BB9A-4245-87B9-43AD57DD821D@cs.otago.ac.nz> On May 13, 2010, at 5:50 PM, Ulf Wiger wrote: > On 05/13/2010 07:09 AM, ??? wrote: >> Hi, I have two questions about pid of remote process: >> >> 1. How do I know if the remote process identified by one PID is >> alive? I >> know I can send a message to it, and receive response with timeout. >> But I >> hope there is some way more conveniently. > > If you monitor it, you will be notified when it dies - or immediately > if it is already dead. Here's a possible scenario. T1: you learn about the existence of a remote process. T2: you monitor it. T3: it dies, and a message is launched to tell you that. T4: is *NOW*. According to the best possible information available to you, the process is still alive. T5: the death notice arrives. We inhabit a universe with a finite speed of light. Learn to live with it. From ulf.wiger@REDACTED Thu May 13 13:48:14 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 May 2010 13:48:14 +0200 Subject: [erlang-questions] Questions about PID of remote process In-Reply-To: <9A99A5EF-BB9A-4245-87B9-43AD57DD821D@cs.otago.ac.nz> References: <4BEB9325.4090806@erlang-solutions.com> <9A99A5EF-BB9A-4245-87B9-43AD57DD821D@cs.otago.ac.nz> Message-ID: <4BEBE6FE.5050105@erlang-solutions.com> Very true. I didn't mean 'immediately' as in the very same period in the oscillation of a Cesium atom. :) In the case of a remote process, obviously a request will have to be sent to the node in question, which can 'immediately' determine if the process is dead, and then send a reply back. This is similar to the situation when you talk to a server, which implies a monitor if you use gen_server:call(). If the call causes an exception due to a 'DOWN' message from the server, you cannot know whether: - the server was down before you even sent the message - the server died while your message was in the queue, waiting to be processed - the server died when it looked at your message - it had time to do most of the processing, possibly involving side-effects, but before it had time to respond. BR, Ulf W On 05/13/2010 01:26 PM, Richard O'Keefe wrote: > > On May 13, 2010, at 5:50 PM, Ulf Wiger wrote: > >> On 05/13/2010 07:09 AM, ??? wrote: >>> Hi, I have two questions about pid of remote process: >>> >>> 1. How do I know if the remote process identified by one PID is alive? I >>> know I can send a message to it, and receive response with timeout. >>> But I >>> hope there is some way more conveniently. >> >> If you monitor it, you will be notified when it dies - or immediately >> if it is already dead. > > Here's a possible scenario. > T1: you learn about the existence of a remote process. > T2: you monitor it. > T3: it dies, and a message is launched to tell you that. > T4: is *NOW*. According to the best possible information > available to you, the process is still alive. > T5: the death notice arrives. > > We inhabit a universe with a finite speed of light. > Learn to live with it. > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From zeno490@REDACTED Thu May 13 17:09:23 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Thu, 13 May 2010 11:09:23 -0400 Subject: [erlang-questions] Current state of packaged modules: net.rtmp.decoder In-Reply-To: References: Message-ID: I agree with 7 not being ideal. From memory Java imposes something similar with packages and I never liked it. However, I could live with it. Also, erlang already imposes a directory structure in other features: for code:lib_dir to work, your beam files (or is it app file?) must be in a directory structure resembling: /.../app-version/ebin If this is not the case, -include_lib("app/include/foo.hrl"). will not work as code:lib_dir fails to find the app package directory. I recently ran into this myself and cursed/cussed quite a bit. Seeing as the evil is already within, it doesn't sound half as bad to let it stay and spread to packages. On Wed, May 12, 2010 at 6:40 PM, Robert Virding wrote: > I quite agree with all Richards comments/critiques and would like to > add one more: > > (7) It imposes a directory structure on packages which I think is > undesirable. I do not want the language to force me to structure my > apps in a special way. Currently all that OTP assumes is that the > source code for a module is in a file called .erl and the > BEAM code is in a file called .beam. This, I think, is > quite enough. It is actually not too difficult to get around the > second one if you are prepared to load code yourself. > > Otherwise I can add '_' in my module names myself. > > -1 > > Robert > > On 12 May 2010 05:52, Richard O'Keefe wrote: > > > > On May 11, 2010, at 9:53 PM, Max Lapshin wrote: > > > >> Hi, I can't understand why is not recommended now to use packages of > >> modules. > > > > There are several important things here. > > > > (1) Erlang allows dots in atoms, always has. foo.bar.ugh > > and foo@REDACTED@ugh and foo_bar_ugh are just plain atoms. > > They are the _same_ atoms as 'foo.bar.ugh', 'foo@REDACTED@ugh', > > and 'foo_bar_ugh'. > > > > This does _not_ apply to Java, where foo_bar is one > > identifier but foo.bar is not. > > > > (2) Historically, the module system treated all atoms alike. > > There was no connection between foo.bar and bar any more > > than there was any connection between foo@REDACTED and bar or > > between foo_bar and bar. In the same way, Java sees no > > connection between a class called Foo_Bar and a class > > called Bar. > > > > One of the fundamental concepts in computer programming > > is alpha-conversion (the idea that names are *arbitrary* > > labels). It is generally a bad idea to violate this. > > > > (3) Formerly, in -module(fred), a call jill:bucket(56) was a > > call to the module 'jill'. Change the -module directive > > to -module(demo.fred) and that call now refers to the > > module 'demo.jill'. This is fine when you want it to > > happen, but if your code referred to existing OTP modules > > such as 'lists', well hello demo.lists breakage. You _can_ > > refer to top level modules using ''.lists or .lists, but > > that's an incompatible change. > > > > (4) It's really not clear how all of this works *dynamically*. > > Before dotted module names got special treatment, > > jill:bucket(56), %A > > M = jill, M:bucket(56), %B > > apply(jill, bucket, [56]), %C > > 'my.module':'my.apply'(jill, bucket, [56]) %D > > were all equivalent, where > > -module('my.module'). > > -export(['my.apply'/3]). > > my.apply(M, F, A) -> apply(M, F, A). > > The abbreviation kluge obviously works for %A. It could > > be made to work for %B by using different (slower!) code > > like M = jill, (fix_mod(M, 'my.')):bucket(56), where > > fix_mod(M, C) -> case has_dots(M) > > of true -> M > > ; false -> atom_append(C, M) > > end. > > It's easy to make %C work when the first argument is a > > constant, like %A, and when the first argument is not a > > constant, the same technique used for %B will work. > > > > But how do you make %D work? > > > > (5) There are no actual packages. There are directories where > > modules with the same prefix will be sought, but it is not > > the case that all files with the same prefix must be in the > > same directory. A "package" is nothing but a bunch of modules > > with a common prefix to their names; there are no package > > properties. > > > > (6) Package names are back to front. This has been explained repeatedly > > in this mailing list. They are like absolute file names, not like > > relative ones. If I need the same package in two places in the > > package hierarchy, I can't do it. I can't move a package up or > > down in the package hierarchy without surgery on every file. I > > cannot make a *component* that my customers can place anywhere > > they want in the package hierarchy. And if I can't do *that*, > > what the heck is the package system *for*? > > > > So: > > - dotted module names was not a new thing > > - mapping dotted names to the file system as described in > > http://www.erlang.se/publications/packages.html > > obviously has its uses, but this could be done with any > > character, including @, or indeed by other means entirely > > (like Eiffel's LACE) > > - the scheme leads to increased work when adopting dotted names > > and makes renaming packages difficult and fragile > > - it does not seem that dynamic behaviour can match the static > > abbreviation facility without increased costs (when known > > functions/forms are involved) or at all (when user defined > > functions are involved) > > > >> What is bad in > >> using dotted names? What is the future of this feature: it will be > >> developed or it will be deprecated? > > > > It should be taken out and shot. > > > > *Some* sort of module connection/configuration facility is needed, > > but I suggest as a design criterion that it should be possible to > > move an entire "flotilla" of modules in the package name space > > by changing *ONE* line in some file. > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Thu May 13 17:29:31 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 13 May 2010 09:29:31 -0600 Subject: fire off a task via erlang Message-ID: Yesterday some colleagues were talking about "how they would keep track of possibly millions of tasks that would need to be fired off on a given interval that users could select." I.e, they were discussing how this could be made possible. For instance a user would log in to a web interface and they would set a task that could be an email or text message that would fire off once or every hour or every day at 1pm, etc. Kind of like cron or even something like google calendar (sending event notifictions). I was telling them that erlang could probably handle something like this. I was thinking about the timer module. The timer module would work for this wouldn't it? I'm not sure how it would scale though. Each user could manage their own timers, it would seem. Anyone have thoughts on this? thx, -wes From mevans@REDACTED Thu May 13 17:34:20 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 13 May 2010 11:34:20 -0400 Subject: [erlang-questions] fire off a task via erlang In-Reply-To: References: Message-ID: You could certainly use the timer module. But there is a small performance overhead. Maybe a better choice would be each task to be its own gen_server and use erlang:send_after(Delay,self(),Message). IRC that has a lower overhead. You would call the first instance of send_after in the init function, and then the message would be caught by handle_info, where you restart the timer. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Wes James Sent: Thursday, May 13, 2010 11:30 AM To: erlang-questions@REDACTED Subject: [erlang-questions] fire off a task via erlang Yesterday some colleagues were talking about "how they would keep track of possibly millions of tasks that would need to be fired off on a given interval that users could select." I.e, they were discussing how this could be made possible. For instance a user would log in to a web interface and they would set a task that could be an email or text message that would fire off once or every hour or every day at 1pm, etc. Kind of like cron or even something like google calendar (sending event notifictions). I was telling them that erlang could probably handle something like this. I was thinking about the timer module. The timer module would work for this wouldn't it? I'm not sure how it would scale though. Each user could manage their own timers, it would seem. Anyone have thoughts on this? thx, -wes ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From comptekki@REDACTED Thu May 13 17:43:48 2010 From: comptekki@REDACTED (Wes James) Date: Thu, 13 May 2010 09:43:48 -0600 Subject: [erlang-questions] fire off a task via erlang In-Reply-To: References: Message-ID: On Thu, May 13, 2010 at 9:34 AM, Evans, Matthew wrote: > You could certainly use the timer module. But there is a small performance overhead. > > Maybe a better choice would be each task to be its own gen_server and use erlang:send_after(Delay,self(),Message). > > IRC that has a lower overhead. You would call the first instance of send_after in the init function, and then the message would be caught by handle_info, where you restart the timer. Matt, I was wondering about erlang's concurrency capabilities too. I've only done some "sequential" erlang. thx, -wes From erlangy@REDACTED Thu May 13 19:05:28 2010 From: erlangy@REDACTED (Michael) Date: Thu, 13 May 2010 10:05:28 -0700 Subject: [erlang-questions] fire off a task via erlang In-Reply-To: References: Message-ID: <20100513170528.GZ4020@delora.autosys.us> On Thu, May 13, 2010 at 09:29:31AM -0600, Wes James wrote: > Yesterday some colleagues were talking about "how they would keep > track of possibly millions of tasks that would need to be fired off on > a given interval that users could select." I.e, they were discussing > how this could be made possible. For instance a user would log in to a > web interface and they would set a task that could be an email or text > message that would fire off once or every hour or every day at 1pm, > etc. Kind of like cron or even something like google calendar > (sending event notifictions). I was telling them that erlang could > probably handle something like this. I was thinking about the timer > module. The timer module would work for this wouldn't it? I'm not > sure how it would scale though. Each user could manage their own > timers, it would seem. Anyone have thoughts on this? > > thx, > > -wes > ________________________________________________________________ maybe ... http://forum.trapexit.org/viewtopic.php?t=15015&sid=8e3c52cb4f7f50d9dcc7c4b52a77aa68 which I have used for very low volume notifications (i.e. one every N minutes) I currently use a simple scan-every-5-minutes gen_server on a couchdb database. It will not scale well as-is (looks through entire table every time). http://www.xquad.com/en/remindmef (or httpS://... if you don't mind self-signed certs) ~M From hd2010@REDACTED Thu May 13 21:43:29 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 13 May 2010 21:43:29 +0200 Subject: Documentation Message-ID: <4BEC5661.1010907@eonblast.com> Does anyone need a volunteer to improve Erlang documentation, especially searchability? I just tried to find information on source documentation* rules and source documentation generation, online. I didn't find it. I reverted to Joe's book. That's nice, in a way, I like paper a lot. But it's also a bit weird. [So if you came here from Google: Joe's book, Appendix A is what you are looking for. Save time, pull it from the shelf. Page 385 it must be. Oh. Wait a minute. Even in Joe's book, the appendix about documentation is hidden on a page without a number. Well, the one after 384 then it is, right? ] Henning * yes I am talking two different kinds of documentation here. I was looking for documentation on source documentation standards and rules. From rapsey@REDACTED Fri May 14 09:05:37 2010 From: rapsey@REDACTED (Rapsey) Date: Fri, 14 May 2010 09:05:37 +0200 Subject: select on tuple in mnesia Message-ID: If I have a tuple saved in an indexed field in mnesia, something like this: -record(mnesia_record,{key,val}) #mnesia_record{key = {atom1,something}, val = "blabla"} Will select call for all keys that are of the {atom1,_} pattern use an index, or will it have to scan the whole table? thank you, Sergej From ulf.wiger@REDACTED Fri May 14 10:02:25 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 14 May 2010 10:02:25 +0200 Subject: [erlang-questions] select on tuple in mnesia In-Reply-To: References: Message-ID: <4BED0391.9040303@erlang-solutions.com> On 05/14/2010 09:05 AM, Rapsey wrote: > If I have a tuple saved in an indexed field in mnesia, something like this: > -record(mnesia_record,{key,val}) > #mnesia_record{key = {atom1,something}, val = "blabla"} > > Will select call for all keys that are of the {atom1,_} pattern use an > index, or will it have to scan the whole table? If the key pattern is constant, it will amount to a lookup if the table is a hash set, or a constrained search if it's an ordered_set. Note that the head pattern must be constant; if you use a dollar variable in the head pattern and constrain it in a guard, it will be a full scan. Ordered sets can be extremely useful with this trick. If you use a hierarchical key, you can do very efficient range queries, by binding the first part of the key. As a silly example, you could have a table keyed on datetime tuples: same_minute({Date, {H,M,_}}) -> HeadPat = #events{time = {Date,{H,M,'_'}}, _='_'}, mnesia:select(mytab, [{HeadPat,[],['$_']}]). ...which could be called as same_minute(erlang:universaltime()). BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From mazen.harake@REDACTED Fri May 14 12:56:13 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 14 May 2010 13:56:13 +0300 Subject: [erlang-questions] fire off a task via erlang In-Reply-To: References: Message-ID: <4BED2C4D.40201@erlang-solutions.com> Hi, The pattern I usually implement for this kind of behaviour is an ETS table which is sorted by the time of the next event and then I poll it every second. If the first element is not triggered then nothing else should trigger either (because it is sorted). When it does trigger I immediately check the next element until I reach an element in the table that is in the future or the end of the table. This works well with repeating events as well because you can just "reschedule" a new event or just insert as many as you want for future use. This might meet your requirements. /Mazen On 13/05/2010 18:43, Wes James wrote: > On Thu, May 13, 2010 at 9:34 AM, Evans, Matthew wrote: > >> You could certainly use the timer module. But there is a small performance overhead. >> >> Maybe a better choice would be each task to be its own gen_server and use erlang:send_after(Delay,self(),Message). >> >> IRC that has a lower overhead. You would call the first instance of send_after in the init function, and then the message would be caught by handle_info, where you restart the timer. >> > Matt, > > I was wondering about erlang's concurrency capabilities too. I've > only done some "sequential" erlang. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From jay@REDACTED Fri May 14 18:15:47 2010 From: jay@REDACTED (Jay Nelson) Date: Fri, 14 May 2010 09:15:47 -0700 Subject: Classification of concurrency patterns Message-ID: <9559FC5B-E9F4-45B6-8388-241E7848A819@duomark.com> Joe's list seems to me to address only one general category: parallel computation. The distinctions in the list may or may not overlap, but they are all computational parallelisms. Other terms for computational concurrency that come to mind include PubSub and Scatter / Gather (which are variants of some that were already listed). The supervisor pattern only works with concurrency. It is an Organizational Pattern rather than a Computational Pattern. I would add the following types of concurrency as a separate category: Organizational Concurrency - Supervisor hierarchies - Serial servers (ala gen_server) - Event-driven reactors (spawn on each event) - Process-striped buffers (gen_stream**) Supervisors detect failure of other processes, therefore they must be implemented using concurrency. The intelligence of system management is distributed among the supervisors and defines the behavior of the system under stress. Serial servers are used to isolate state and provide distributed asynchronous access to state (or a computational service) for a series of unrelated clients. Servers also can be used to impose a serial order on events, or provide distributed rendezvous control. Event-driven concurrency is used in web servers so that there is no contagion of errors from one client to another. It also ensures memory leaks are not likely to affect the system since spawned requests are short-lived. Any system which has variable response time in reaction to events can benefit from concurrency by overlapping computations in time. **This weekend I will be checking in gen_stream which uses process- striped buffering as a new example of organizational concurrency to provide efficient access to a slow data stream by prefetching the data. A large piece of data is segmented across processes in a round- robin fashion much like RAID uses striping of disks. jay From enewhuis@REDACTED Fri May 14 19:22:20 2010 From: enewhuis@REDACTED (Eric Newhuis) Date: Fri, 14 May 2010 12:22:20 -0500 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: <9559FC5B-E9F4-45B6-8388-241E7848A819@duomark.com> References: <9559FC5B-E9F4-45B6-8388-241E7848A819@duomark.com> Message-ID: Brilliant. On May 14, 2010, at 11:15 AM, Jay Nelson wrote: > Joe's list seems to me to address only one general category: > parallel computation. The distinctions in the list may or may not > overlap, but they are all computational parallelisms. Other terms > for computational concurrency that come to mind include PubSub and > Scatter / Gather (which are variants of some that were already > listed). > > The supervisor pattern only works with concurrency. It is an > Organizational Pattern rather than a Computational Pattern. > > I would add the following types of concurrency as a separate category: > > Organizational Concurrency > - Supervisor hierarchies > - Serial servers (ala gen_server) > - Event-driven reactors (spawn on each event) > - Process-striped buffers (gen_stream**) > > Supervisors detect failure of other processes, therefore they must > be implemented using concurrency. The intelligence of system > management is distributed among the supervisors and defines the > behavior of the system under stress. > > Serial servers are used to isolate state and provide distributed > asynchronous access to state (or a computational service) for a > series of unrelated clients. Servers also can be used to impose a > serial order on events, or provide distributed rendezvous control. > > Event-driven concurrency is used in web servers so that there is no > contagion of errors from one client to another. It also ensures > memory leaks are not likely to affect the system since spawned > requests are short-lived. Any system which has variable response > time in reaction to events can benefit from concurrency by > overlapping computations in time. > > **This weekend I will be checking in gen_stream which uses process- > striped buffering as a new example of organizational concurrency to > provide efficient access to a slow data stream by prefetching the > data. A large piece of data is segmented across processes in a > round-robin fashion much like RAID uses striping of disks. > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From enewhuis@REDACTED Fri May 14 21:05:44 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Fri, 14 May 2010 14:05:44 -0500 Subject: I think I wish I could write case Any of whatever -> _ end. Message-ID: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> Consider: X = case Any of very_long_pattern_or_whatever -> very_long_pattern_or_whatever end. I'd rather abbreviate that like this. X = case Any of very_long_pattern_or_whatever -> _ end. It also seems more readable to me than: X = case Any of very_long_pattern_or_whatever=Y -> Y end. From zeno490@REDACTED Fri May 14 21:13:09 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Fri, 14 May 2010 15:13:09 -0400 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> Message-ID: This looks like an assert to me? You just want to ensure that Any is matching the very long pattern, correct? In that case, you may be better off with an assert macro, something like: ?ASSERT_MATCH(Any, very_long_pattern_or_whatever). And write ASSERT_MATCH yourself. You can also write it as: very_long_pattern_or_whatever = Any. Although guards may not work in the later. You can also make it a single function that you'll call: ensure_whatever(Any) when some_guards -> very_long_pattern_or_whatever = Any. That way you might get better debugging info if what you pass to ensure_whatever does not match. 2cents On Fri, May 14, 2010 at 3:05 PM, Eric Newhuis (personal) wrote: > Consider: > > X = case Any of > very_long_pattern_or_whatever -> > very_long_pattern_or_whatever > end. > > > I'd rather abbreviate that like this. > > X = case Any of > very_long_pattern_or_whatever -> _ > end. > > It also seems more readable to me than: > > X = case Any of > very_long_pattern_or_whatever=Y -> Y > end. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rvirding@REDACTED Fri May 14 22:11:20 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 14 May 2010 22:11:20 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> Message-ID: The trouble with this is that _ already has a very specific meaning, it s the don't care variable. It will always match anything and never be bound and it is used as a place holder when you know there is something there but you don't care what it is. Using it in a context where it gets a value would cause much confusion and lead to funny inconsistent cases. Robert On 14 May 2010 21:05, Eric Newhuis (personal) wrote: > Consider: > > X = case Any of > ? ? ? ? ? ? ? ?very_long_pattern_or_whatever -> > ? ? ? ? ? ? ? ? ? ? ? ?very_long_pattern_or_whatever > ? ? ? ?end. > > > I'd rather abbreviate that like this. > > X = case Any of > ? ? ? ?very_long_pattern_or_whatever -> _ > ? ? ? ?end. > > It also seems more readable to me than: > > X = case Any of > ? ? ? ?very_long_pattern_or_whatever=Y -> Y > ? ? ? ?end. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jeedward@REDACTED Fri May 14 21:22:09 2010 From: jeedward@REDACTED (John Edward) Date: Fri, 14 May 2010 12:22:09 -0700 (PDT) Subject: Call for papers : MULTICONF-10, USA, July 2010 Message-ID: <100901.14189.qm@web45911.mail.sp1.yahoo.com> It would be highly appreciated if you could share this announcement with your colleagues, students and individuals whose research is in computer science, computer engineering, information science and related areas. Call for papers : MULTICONF-10, USA, July 2010 The 2010 multi-conference (MULTICONF-10) (website: http://www.PromoteResearch.org ) will be held during July 12-14, 2010 in Orlando, Florida, USA. The primary goal of MULTICONF is to promote research and developmental activities in computer science, information technology, control engineering, and related fields. Another goal is to promote the dissemination of research to a multidisciplinary audience and to facilitate communication among researchers, developers, practitioners in different fields. The following conferences are planned to be organized as part of MULTICONF-10. ? International Conference on Artificial Intelligence and Pattern Recognition (AIPR-10) ? International Conference on Automation, Robotics and Control Systems (ARCS-10) ? International Conference on Bioinformatics, Computational Biology, Genomics and Chemoinformatics (BCBGC-10) ? International Conference on Computer Communications and Networks (CCN-10) ? International Conference on Enterprise Information Systems and Web Technologies (EISWT-10) ? International Conference on High Performance Computing Systems (HPCS-10) ? International Conference on Information Security and Privacy (ISP-10) ? International Conference on Image and Video Processing and Computer Vision (IVPCV-10) ? International Conference on Software Engineering Theory and Practice (SETP-10) ? International Conference on Theoretical and Mathematical Foundations of Computer Science (TMFCS-10) MULTICONF-10 will be held at Imperial Swan Hotel and Suites. It is a full-service resort that puts you in the middle of the fun! Located 1/2 block south of the famed International Drive, the hotel is just minutes from great entertainment like Walt Disney World? Resort, Universal Studios and Sea World Orlando. Guests can enjoy free scheduled transportation to these theme parks, as well as spacious accommodations, outdoor pools and on-site dining ? all situated on 10 tropically landscaped acres. Here, guests can experience a full-service resort with discount hotel pricing in Orlando. Please see the website http://www.PromoteResearch.org for more details. Sincerely John Edward From comptekki@REDACTED Fri May 14 23:25:52 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 14 May 2010 15:25:52 -0600 Subject: how to escape ? character Message-ID: How do I escape ? in a string. The runtime things it is an illegal macro call. Even if I do \?. -wes From koops.j@REDACTED Fri May 14 23:32:36 2010 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 14 May 2010 23:32:36 +0200 Subject: [erlang-questions] how to escape ? character In-Reply-To: References: Message-ID: "How do I escape " ++ [63] ++ " in a string" ++ [63]. On Fri, May 14, 2010 at 11:25 PM, Wes James wrote: > How do I escape ? in a string. The runtime things it is an illegal > macro call. Even if I do \?. > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Fri May 14 23:36:54 2010 From: comptekki@REDACTED (Wes James) Date: Fri, 14 May 2010 15:36:54 -0600 Subject: [erlang-questions] how to escape ? character In-Reply-To: References: Message-ID: On Fri, May 14, 2010 at 3:32 PM, Jeroen Koops wrote: > "How do I escape " ++ [63] ++ " in a string" ++ [63]. > > On Fri, May 14, 2010 at 11:25 PM, Wes James wrote: >> >> How do I escape ? in a string. ?The runtime things it is an illegal >> macro call. ?Even if I do \?. >> Jeroen, Thx. As I was looking on erlang.org in the docs for this - I was also looking for ord and char like functions. What are they called and under what module? I'm looking in the stdlib, but can't see them anywhere. i.e. ord("m") -> integer value and char(integer value) -> some letter. thx, -wes From koops.j@REDACTED Fri May 14 23:52:11 2010 From: koops.j@REDACTED (Jeroen Koops) Date: Fri, 14 May 2010 23:52:11 +0200 Subject: [erlang-questions] how to escape ? character In-Reply-To: References: Message-ID: Hi Wes, In Erlang, a string is simply a list of integers that are all valid ASCII values. The expressions "hello" and [ 104, 101, 108, 108, 111 ] are exactly the same. The shell (and some of the the format modifiers in io:format) has some intelligence - if it has to print a list that seems to consist mostly of ASCII values, it will display it as a double-quoted string - otherwise, it will display it as a list-in-square-brackets. As such, there's no need for ord and char functions in Erlang. If you want to print the value of say ASCII character 42, simply do something like this:io:format("~s", [ [42] ]). If you want to print the ASCII values of all characters in the string "hello", simply do something like this: lists:foreach(fun(C) -> io:format("~b~n", [C]) end, "hello"). On Fri, May 14, 2010 at 11:36 PM, Wes James wrote: > On Fri, May 14, 2010 at 3:32 PM, Jeroen Koops wrote: > > "How do I escape " ++ [63] ++ " in a string" ++ [63]. > > > > On Fri, May 14, 2010 at 11:25 PM, Wes James wrote: > >> > >> How do I escape ? in a string. The runtime things it is an illegal > >> macro call. Even if I do \?. > >> > > > Jeroen, > > Thx. As I was looking on erlang.org in the docs for this - I was also > looking for ord and char like functions. What are they called and > under what module? I'm looking in the stdlib, but can't see them > anywhere. i.e. ord("m") -> integer value and char(integer value) -> > some letter. > > thx, > > -wes > From hd2010@REDACTED Sat May 15 00:03:31 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 15 May 2010 00:03:31 +0200 Subject: Shorten output Message-ID: <4BEDC8B3.8070205@eonblast.com> Hello list, I would like to shorten any output to certain limit. ~W and ~P won't do. I came up with the following but I am sure there is a much better solution. Thanks, Henning %%%---------------------------------------------------------------------------- %%% cut any io:format parameters short %%%---------------------------------------------------------------------------- -define(DELIMIT, 100). delimited([]) -> []; delimited(List) when is_list(List), length(List) > ?DELIMIT -> delimited(lists:append(lists:sublist(List, ?DELIMIT),"...")); delimited(List) when is_list(List) -> [ delimited(X) || X <- List ]; delimited(Tuple) when is_tuple(Tuple) -> list_to_tuple([ delimited(X) || X <- tuple_to_list(Tuple) ]); delimited(Item) when is_binary(Item), size(Item) > ?DELIMIT -> <> = Item, <>; delimited(Item) -> Item. From brady.mccary@REDACTED Sat May 15 01:08:10 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Fri, 14 May 2010 18:08:10 -0500 Subject: ODBC Crashing Message-ID: erlang-questions, The odbc port process is crashing. The system is erlang R13B04 on windows. Database is MSSQL. I am almost certain that the crash has to do with parsing/marshaling of decimal types, e.g., Sql = "exec dbo.foo bar = ?", Bar = {{sql_decimal, 19, 6}, "123.456"}, odbc:param_query(Ref, Sql, [Bar]) Note that precision is greater or equal to 16 so that the odbc port driver will probably be handling them as strings (according to the documentation). The crash is intermittent, i.e., if Bar is fixed, then the call will succeed more often than not. The need is not pressing b/c we have switched from using {sql_decimal, P, S} to {sql_varchar, N} b/c MSSQL parses the decimal out of the varchar fine for our purposes. Brady From hd2010@REDACTED Sat May 15 02:02:58 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 15 May 2010 02:02:58 +0200 Subject: [erlang-questions] ODBC Crashing In-Reply-To: References: Message-ID: <4BEDE4B2.3010502@eonblast.com> Brady, could you give me a tip on where to find the sources you are talking about, if they are open? I'd like to take a look how they handle decimals. Thanks, Henning Brady McCary wrote: > erlang-questions, > > The odbc port process is crashing. The system is erlang R13B04 on > windows. Database is MSSQL. I am almost certain that the crash has to > do with parsing/marshaling of decimal types, e.g., > > Sql = "exec dbo.foo bar = ?", > Bar = {{sql_decimal, 19, 6}, "123.456"}, > odbc:param_query(Ref, Sql, [Bar]) > > Note that precision is greater or equal to 16 so that the odbc port > driver will probably be handling them as strings (according to the > documentation). The crash is intermittent, i.e., if Bar is fixed, then > the call will succeed more often than not. The need is not pressing > b/c we have switched from using {sql_decimal, P, S} to {sql_varchar, > N} b/c MSSQL parses the decimal out of the varchar fine for our > purposes. > > Brady > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From catsunny2010@REDACTED Sat May 15 02:41:15 2010 From: catsunny2010@REDACTED (catsunny2010) Date: Sat, 15 May 2010 08:41:15 +0800 Subject: That's why? Message-ID: <1273884075.12447.5.camel@qianqian-m2gta> Hi all, I know I can, <> = <<..>> and I can, <> = <<...>> But why cannot I, <> = <<...>> ??? Thank you! catsunny2010 From brady.mccary@REDACTED Sat May 15 03:51:52 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Fri, 14 May 2010 20:51:52 -0500 Subject: [erlang-questions] ODBC Crashing In-Reply-To: <4BEDE4B2.3010502@eonblast.com> References: <4BEDE4B2.3010502@eonblast.com> Message-ID: Henning, Unfortunately they are not open. However, if you (or another) will link me to general instructions on how to debug erlang port processes I will be glad to find the issue myself in off-time and post the patch to this list. Brady On Fri, May 14, 2010 at 7:02 PM, Henning Diedrich wrote: > Brady, could you give me a tip on where to find the sources you are talking > about, if they are open? > > I'd like to take a look how they handle decimals. > > Thanks, > Henning > > Brady McCary wrote: >> >> erlang-questions, >> >> The odbc port process is crashing. The system is erlang R13B04 on >> windows. Database is MSSQL. I am almost certain that the crash has to >> do with parsing/marshaling of decimal types, e.g., >> >> Sql = "exec dbo.foo bar = ?", >> Bar = {{sql_decimal, 19, 6}, "123.456"}, >> odbc:param_query(Ref, Sql, [Bar]) >> >> Note that precision is greater or equal to 16 so that the odbc port >> driver will probably be handling them as strings (according to the >> documentation). The crash is intermittent, i.e., if Bar is fixed, then >> the call will succeed more often than not. The need is not pressing >> b/c we have switched from using {sql_decimal, P, S} to {sql_varchar, >> N} b/c MSSQL parses the decimal out of the varchar fine for our >> purposes. >> >> Brady >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > From brady.mccary@REDACTED Sat May 15 03:51:52 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Fri, 14 May 2010 20:51:52 -0500 Subject: [erlang-questions] ODBC Crashing In-Reply-To: <4BEDE4B2.3010502@eonblast.com> References: <4BEDE4B2.3010502@eonblast.com> Message-ID: Henning, Unfortunately they are not open. However, if you (or another) will link me to general instructions on how to debug erlang port processes I will be glad to find the issue myself in off-time and post the patch to this list. Brady On Fri, May 14, 2010 at 7:02 PM, Henning Diedrich wrote: > Brady, could you give me a tip on where to find the sources you are talking > about, if they are open? > > I'd like to take a look how they handle decimals. > > Thanks, > Henning > > Brady McCary wrote: >> >> erlang-questions, >> >> The odbc port process is crashing. The system is erlang R13B04 on >> windows. Database is MSSQL. I am almost certain that the crash has to >> do with parsing/marshaling of decimal types, e.g., >> >> Sql = "exec dbo.foo bar = ?", >> Bar = {{sql_decimal, 19, 6}, "123.456"}, >> odbc:param_query(Ref, Sql, [Bar]) >> >> Note that precision is greater or equal to 16 so that the odbc port >> driver will probably be handling them as strings (according to the >> documentation). The crash is intermittent, i.e., if Bar is fixed, then >> the call will succeed more often than not. The need is not pressing >> b/c we have switched from using {sql_decimal, P, S} to {sql_varchar, >> N} b/c MSSQL parses the decimal out of the varchar fine for our >> purposes. >> >> Brady >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > From hd2010@REDACTED Sat May 15 04:25:57 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 15 May 2010 04:25:57 +0200 Subject: [erlang-questions] That's why? In-Reply-To: <1273884075.12447.5.camel@qianqian-m2gta> References: <1273884075.12447.5.camel@qianqian-m2gta> Message-ID: <4BEE0635.5010308@eonblast.com> I may fancy that 3+2 is optimized to a literal number 5 at compile time, which is as unbreakable as the plain reference to A in the first example. Doing anything more complex may not have been reconcilable with the optimizations applied to pattern matching. That's how I am comforting myself when running into little amendable guards and this very issue of byte sizes you are raising. Henning catsunny2010 wrote: > Hi all, > > I know I can, > > <> = <<..>> > > and I can, > > <> = <<...>> > > But why cannot I, > > <> = <<...>> ??? > > Thank you! > catsunny2010 > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From wiener.guy@REDACTED Sat May 15 10:49:18 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Sat, 15 May 2010 11:49:18 +0300 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: Hello Joe, and everyone on the Erlang list, A colleague of mine commented that this list contains only transactional patterns (i.e, take data, process it and output it), but no reactive patterns (i.e, a system that responds to events over time without a single output or a predefined point of termination). One can observe several patterns for parallel reactive architectures, namely by the way the process collaborate: 1) Clique - Every process communicate with almost any other process. Each message includes the sender, to distinguish from other messages. 2) Broadcast - Processes do not collaborate with other specific processes, but broadcast messages without expecting a specific reply. 3) Subscribe/Notify - Processes broadcast messages to other pre-subscribed processes. 4) Star - "Peripheral" processes collaborate through a small group (potentially just one) of "central" processes, who decide for the rest what to do. One can also observe between two kinds of parallel reactive behavioral protocols: 1) Independent - Each process decides for itself what is the next step, regardless of other processes. 2) Consensus - All processes try to reach some agreement on the next step (often using a broadcast or star pattern). Would you consider these to be parallel patterns? Best, Guy Wiener. From mazen.harake@REDACTED Sat May 15 12:47:59 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Sat, 15 May 2010 13:47:59 +0300 Subject: [erlang-questions] how to escape ? character In-Reply-To: References: Message-ID: <4BEE7BDF.9000803@erlang-solutions.com> One can also use the notation $ which is slightly more comfortable when it comes to individual chars. Try e.g this in the shell: 1> $?. 63 Note however that this is not an Erlang string! $? is not a string, [$?] however is. /M On 15/05/2010 00:52, Jeroen Koops wrote: > Hi Wes, > > In Erlang, a string is simply a list of integers that are all valid ASCII > values. The expressions "hello" and [ 104, 101, 108, 108, 111 ] are exactly > the same. > The shell (and some of the the format modifiers in io:format) has some > intelligence - if it has to print a list that seems to consist mostly of > ASCII values, it will display it as a double-quoted string - otherwise, it > will display it as a list-in-square-brackets. > > As such, there's no need for ord and char functions in Erlang. If you want > to print the value of say ASCII character 42, simply do something like > this:io:format("~s", [ [42] ]). > If you want to print the ASCII values of all characters in the string > "hello", simply do something like this: lists:foreach(fun(C) -> > io:format("~b~n", [C]) end, "hello"). > > On Fri, May 14, 2010 at 11:36 PM, Wes James wrote: > > >> On Fri, May 14, 2010 at 3:32 PM, Jeroen Koops wrote: >> >>> "How do I escape " ++ [63] ++ " in a string" ++ [63]. >>> >>> On Fri, May 14, 2010 at 11:25 PM, Wes James wrote: >>> >>>> How do I escape ? in a string. The runtime things it is an illegal >>>> macro call. Even if I do \?. >>>> >>>> >> >> Jeroen, >> >> Thx. As I was looking on erlang.org in the docs for this - I was also >> looking for ord and char like functions. What are they called and >> under what module? I'm looking in the stdlib, but can't see them >> anywhere. i.e. ord("m") -> integer value and char(integer value) -> >> some letter. >> >> thx, >> >> -wes >> >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rvirding@REDACTED Sat May 15 13:30:46 2010 From: rvirding@REDACTED (Robert Virding) Date: Sat, 15 May 2010 13:30:46 +0200 Subject: [erlang-questions] how to escape ? character In-Reply-To: <4BEE7BDF.9000803@erlang-solutions.com> References: <4BEE7BDF.9000803@erlang-solutions.com> Message-ID: 2010/5/15 Mazen Harake : > One can also use the notation $ which is slightly more comfortable when > it comes to individual chars. > > Try e.g this in the shell: > > 1> $?. > 63 > > Note however that this is not an Erlang string! > $? is not a string, [$?] however is. Or even simpler for a string, "?". They are equivalent "?" <=> [$?] <=> [63]. Robert From roberto@REDACTED Sat May 15 13:56:01 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 15 May 2010 13:56:01 +0200 Subject: non-linear behavior? Message-ID: dear list, while performing some optimization tests i found a behavior which i cannot understand. i've stripped it down to a very simple test module. here it is: %%%%%%%%%%%%%%%%%%%%%%%%% -module(lintest). -compile(export_all). start() -> C1 = 100000000, C2 = 1000000000, {M1, _} = timer:tc(?MODULE, decrease, [C1]), {M2, _} = timer:tc(?MODULE, decrease, [C2]), io:format("decrease from ~p took ~p seconds.",[C1, M1/1000000]), io:format("decrease from ~p took ~p seconds.",[C2, M2/1000000]). % computational processes decrease(0) -> ok; decrease(N) -> decrease(N - 1). %%%%%%%%%%%%%%%%%%%%%%%%% if you run this module, you'll get: roberto$ erl Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:true] Eshell V5.7.5 (abort with ^G) 1> c(lintest). {ok,lintest} 2> lintest:start(). decrease from 100000000 took 0.813727 seconds. decrease from 1000000000 took 46.831207 seconds. ok 3> this yields the same results with/without kernel poll, and smp disabled/enabled. i might be missing a point here, but shouldn't this be a linear computation? if it took 0.8 seconds to count down from 100,000,000 shouldn't it take around 10 times more to count down from 1,000,000,000, i.e. around 8 seconds? the resulting 46 seconds seems way strange to me. any kind soul can explain me what i am missing here? thank you, r. From kiszl@REDACTED Sat May 15 14:06:18 2010 From: kiszl@REDACTED (Zoltan Lajos Kis) Date: Sat, 15 May 2010 14:06:18 +0200 Subject: [erlang-questions] non-linear behavior? In-Reply-To: References: Message-ID: <4BEE8E3A.4030200@tmit.bme.hu> On 5/15/2010 1:56 PM, Roberto Ostinelli wrote: > dear list, > > while performing some optimization tests i found a behavior which i > cannot understand. i've stripped it down to a very simple test module. > > here it is: > > %%%%%%%%%%%%%%%%%%%%%%%%% > > -module(lintest). > -compile(export_all). > > start() -> > C1 = 100000000, > C2 = 1000000000, > {M1, _} = timer:tc(?MODULE, decrease, [C1]), > {M2, _} = timer:tc(?MODULE, decrease, [C2]), > io:format("decrease from ~p took ~p seconds.",[C1, M1/1000000]), > io:format("decrease from ~p took ~p seconds.",[C2, M2/1000000]). > > % computational processes > decrease(0) -> ok; > decrease(N) -> decrease(N - 1). > > %%%%%%%%%%%%%%%%%%%%%%%%% > > if you run this module, you'll get: > > > roberto$ erl > Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:true] > > Eshell V5.7.5 (abort with ^G) > 1> c(lintest). > {ok,lintest} > 2> lintest:start(). > decrease from 100000000 took 0.813727 seconds. > decrease from 1000000000 took 46.831207 seconds. > ok > 3> > > this yields the same results with/without kernel poll, and smp disabled/enabled. > > i might be missing a point here, but shouldn't this be a linear > computation? if it took 0.8 seconds to count down from 100,000,000 > shouldn't it take around 10 times more to count down from > 1,000,000,000, i.e. around 8 seconds? the resulting 46 seconds seems > way strange to me. > > any kind soul can explain me what i am missing here? > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > A probable explanation is that integers longer than 28 bits (134,217,727) are represented, and probably handled differently from those below this limit. Zoltan. From roberto@REDACTED Sat May 15 14:24:41 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 15 May 2010 14:24:41 +0200 Subject: [erlang-questions] non-linear behavior? In-Reply-To: <4BEE8E3A.4030200@tmit.bme.hu> References: <4BEE8E3A.4030200@tmit.bme.hu> Message-ID: 2010/5/15 Zoltan Lajos Kis : > > A probable explanation is that integers longer than 28 bits (134,217,727) > are represented, and probably handled differently from those below this > limit. > > Zoltan. thank you zoltan, this is it, and here below is proof. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(lintest2). -compile(export_all). start() -> C = 100000000, T1 = 1, T2 = 10, {M1, _} = timer:tc(?MODULE, decrease, [C, T1, C]), {M2, _} = timer:tc(?MODULE, decrease, [C, T2, C]), io:format("decrease from ~p for ~p times took ~p seconds.~n",[C, T1, M1/1000000]), io:format("decrease from ~p for ~p times took ~p seconds.~n",[C, T2, M2/1000000]). % computational processes decrease(0, 0, _Start) -> ok; decrease(0, Times, Start) -> decrease(Start, Times - 1, Start); decrease(N, Times, Start) -> decrease(N - 1, Times, Start). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% roberto$ erl Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> c(lintest2). {ok,lintest2} 2> lintest2:start(). decrease from 100000000 for 1 times took 1.546736 seconds. decrease from 100000000 for 10 times took 8.472292 seconds. ok 3> r. From mazen.harake@REDACTED Sat May 15 14:25:18 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Sat, 15 May 2010 15:25:18 +0300 Subject: [erlang-questions] how to escape ? character In-Reply-To: References: <4BEE7BDF.9000803@erlang-solutions.com> Message-ID: <4BEE92AE.6040809@erlang-solutions.com> Obviously :) On 15/05/2010 14:30, Robert Virding wrote: > 2010/5/15 Mazen Harake: > >> One can also use the notation $ which is slightly more comfortable when >> it comes to individual chars. >> >> Try e.g this in the shell: >> >> 1> $?. >> 63 >> >> Note however that this is not an Erlang string! >> $? is not a string, [$?] however is. >> > Or even simpler for a string, "?". They are equivalent "?"<=> [$?] > <=> [63]. > > Robert > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From roberto@REDACTED Sat May 15 14:50:18 2010 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 15 May 2010 14:50:18 +0200 Subject: multicore performance - smp Message-ID: dear list, i'm performing some tests to optimize multicore usage. what i want to understand is the best way to use the 4 cores that my machine has, in a very simple erlang computation. i've written this module which basically spawns a very small number of processes which do a trivial but intensive computation, and report to a registered 'counter' process when done. here's the test module: %%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(mtest). -compile(export_all). -define(NUMCOMPUTE, 100000000). -define(NUMTIMES, 10). -define(TIMEOUT, 60000). start(ProcNum) -> % start counter register(counter, spawn(fun() -> counter_loop(now(), ProcNum, ProcNum) end)), % start compute processes [spawn(?MODULE, increase, [0, 0]) || _K <- lists:seq(1, ProcNum)]. % computational processes increase(?NUMCOMPUTE, ?NUMTIMES) -> counter ! finished; increase(?NUMCOMPUTE, Times) -> increase(0, Times + 1); increase(N, Times) -> increase(N + 1, Times). % counter loop counter_loop(Start, 0, ProcNum) -> T = timer:now_diff(now(), Start), io:format("COMPUTED ADD TO ~p FOR ~p TIMES IN ~p PROCESS(ES) IN ~p SECONDS~n", [?NUMCOMPUTE, ?NUMTIMES, ProcNum, T/1000000]); counter_loop(Start, ProcsLeft, ProcNum) -> receive finished -> counter_loop(Start, ProcsLeft - 1, ProcNum); _ -> counter_loop(Start, ProcsLeft, ProcNum) after ?TIMEOUT -> timeout end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%% when i run this module with kernel poll and smp enabled, i get: roberto$ erl +K true Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:true] Eshell V5.7.5 (abort with ^G) 1> c(mtest). {ok,mtest} 2> mtest:start(1). [<0.40.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 1 PROCESS(ES) IN 8.100901 SECONDS 3> mtest:start(2). [<0.43.0>,<0.44.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 2 PROCESS(ES) IN 10.367364 SECONDS 4> mtest:start(3). [<0.47.0>,<0.48.0>,<0.49.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 3 PROCESS(ES) IN 13.541443 SECONDS 5> mtest:start(4). [<0.52.0>,<0.53.0>,<0.54.0>,<0.55.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 4 PROCESS(ES) IN 17.512607 SECONDS 6> i would have expected that, since 1 process running on 1 core takes around 8 seconds, running the code on 4 processes [hence on 4 cores] would have taken only marginally more time [for smp, startup, etc]. however, i see here that 4 cores are taking twice as much time that i would have expected. in this kind of situations, as reported also in other topics, it seems that using 4 erlang instances with smp disabled would definitely allow me to run the very same test in 8 seconds, not in 17. i'm prepared to loose a little for smp, which is normal, but adding twice up is definitely a high cost. i would have thought that this kind of parallelism would have been handled with no hassle in erlang: am i approaching this problem in a wrong manner? what should i do instead to achieve my expected results? thank you, r. From mononcqc@REDACTED Sat May 15 16:14:26 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 15 May 2010 10:14:26 -0400 Subject: [erlang-questions] multicore performance - smp In-Reply-To: References: Message-ID: Having 4 processes doesn't mean they'll all be on different cores. The VM already runs maybe about 20-30 of them to begin with. You should see better concurrent behaviour if you were to do the test with something like 50 processes on 4 cores versus 50 processes on a single one. There are other variables coming into play here, but in general, more processes will make the behaviour better than fewer processes. The VM is meant to run thousands and thousands of processes concurrently in larger systems. Small benchmarks like this are likely to show weird results -- it's not exactly what Erlang would be optimized for. On Sat, May 15, 2010 at 8:50 AM, Roberto Ostinelli wrote: > dear list, > > i'm performing some tests to optimize multicore usage. what i want to > understand is the best way to use the 4 cores that my machine has, in > a very simple erlang computation. > > i've written this module which basically spawns a very small number of > processes which do a trivial but intensive computation, and report to > a registered 'counter' process when done. > > here's the test module: > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > -module(mtest). > -compile(export_all). > -define(NUMCOMPUTE, 100000000). > -define(NUMTIMES, 10). > -define(TIMEOUT, 60000). > > start(ProcNum) -> > % start counter > register(counter, spawn(fun() -> counter_loop(now(), ProcNum, > ProcNum) end)), > % start compute processes > [spawn(?MODULE, increase, [0, 0]) || _K <- lists:seq(1, ProcNum)]. > > % computational processes > increase(?NUMCOMPUTE, ?NUMTIMES) -> counter ! finished; > increase(?NUMCOMPUTE, Times) -> increase(0, Times + 1); > increase(N, Times) -> increase(N + 1, Times). > > % counter loop > counter_loop(Start, 0, ProcNum) -> > T = timer:now_diff(now(), Start), > io:format("COMPUTED ADD TO ~p FOR ~p TIMES IN ~p PROCESS(ES) IN ~p > SECONDS~n", [?NUMCOMPUTE, ?NUMTIMES, ProcNum, T/1000000]); > counter_loop(Start, ProcsLeft, ProcNum) -> > receive > finished -> > counter_loop(Start, ProcsLeft - 1, ProcNum); > _ -> > counter_loop(Start, ProcsLeft, ProcNum) > after ?TIMEOUT -> timeout > end. > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > when i run this module with kernel poll and smp enabled, i get: > > > roberto$ erl +K true > Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:true] > > Eshell V5.7.5 (abort with ^G) > 1> c(mtest). > {ok,mtest} > 2> mtest:start(1). > [<0.40.0>] > COMPUTED ADD TO 100000000 FOR 10 TIMES IN 1 PROCESS(ES) IN 8.100901 SECONDS > 3> mtest:start(2). > [<0.43.0>,<0.44.0>] > COMPUTED ADD TO 100000000 FOR 10 TIMES IN 2 PROCESS(ES) IN 10.367364 > SECONDS > 4> mtest:start(3). > [<0.47.0>,<0.48.0>,<0.49.0>] > COMPUTED ADD TO 100000000 FOR 10 TIMES IN 3 PROCESS(ES) IN 13.541443 > SECONDS > 5> mtest:start(4). > [<0.52.0>,<0.53.0>,<0.54.0>,<0.55.0>] > COMPUTED ADD TO 100000000 FOR 10 TIMES IN 4 PROCESS(ES) IN 17.512607 > SECONDS > 6> > > > i would have expected that, since 1 process running on 1 core takes > around 8 seconds, running the code on 4 processes [hence on 4 cores] > would have taken only marginally more time [for smp, startup, etc]. > however, i see here that 4 cores are taking twice as much time that i > would have expected. > > in this kind of situations, as reported also in other topics, it seems > that using 4 erlang instances with smp disabled would definitely allow > me to run the very same test in 8 seconds, not in 17. i'm prepared to > loose a little for smp, which is normal, but adding twice up is > definitely a high cost. > > i would have thought that this kind of parallelism would have been > handled with no hassle in erlang: am i approaching this problem in a > wrong manner? what should i do instead to achieve my expected results? > > thank you, > > r. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hd2010@REDACTED Sat May 15 16:29:52 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 15 May 2010 16:29:52 +0200 Subject: [erlang-questions] Classification of concurrency patterns In-Reply-To: References: Message-ID: <4BEEAFE0.1050802@eonblast.com> Joe's patterns had one, #5/Grid, with any communication between jobs. The general train of thought starts out purely functional there I guess with communication between jobs on the other end of the scale. Maybe noting the difference between functional and non-functional "jobs" as underlying building blocks of a pattern can clarify thinking. You could have functional jobs, broadcasting jobs, receiving jobs, implicitly and overtly blocking jobs. The latter may be a good early indicator of anti-patterns were they appear in Joe's computing patterns, while being potentially the norm in Guy's scenarios. They'd wait for input to react to. I can think of patterns where all processes are connected to out-of-system resources and receiving signals from there, to then act. Others, e.g. simulations, will receive no input on the part of the jobs but still be non-functional in that they communicate among each other as prescribed in the Grid or in the list below. From Guy's list, all of Joe's, except the Grid, seem to be Stars. The Grid itself seems to belong on Guy's list because of inter-job communication but then again not because it is not meant to be outside-world-reactional as far as I understand it. Jobs - functional - broadcasting - receiving (and possibly broadcasting) - blocking - - - - - - - - - pure computation (all input state is available before start) - receiving from other jobs - receiving (even indirectly) from out of system Henning Guy Wiener wrote: > Hello Joe, and everyone on the Erlang list, > A colleague of mine commented that this list contains only transactional > patterns (i.e, take data, process it and output it), but no reactive > patterns (i.e, a system that responds to events over time without a single > output or a predefined point of termination). > > One can observe several patterns for parallel reactive architectures, namely > by the way the process collaborate: > > 1) Clique - Every process communicate with almost any other process. Each > message includes the sender, to distinguish from other messages. > 2) Broadcast - Processes do not collaborate with other specific processes, > but broadcast messages without expecting a specific reply. > 3) Subscribe/Notify - Processes broadcast messages to other pre-subscribed > processes. > 4) Star - "Peripheral" processes collaborate through a small group > (potentially just one) of "central" processes, who decide for the rest what > to do. > > One can also observe between two kinds of parallel reactive behavioral > protocols: > > 1) Independent - Each process decides for itself what is the next step, > regardless of other processes. > 2) Consensus - All processes try to reach some agreement on the next step > (often using a broadcast or star pattern). > > Would you consider these to be parallel patterns? > > Best, > Guy Wiener. > > From mononcqc@REDACTED Sat May 15 16:59:14 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 15 May 2010 10:59:14 -0400 Subject: [erlang-questions] multicore performance - smp In-Reply-To: References: Message-ID: Just to come back with some numbers: If I run it once here: Erlang R13B04 (erts-5.7.5) [smp:4:4] [rq:4] [async-threads:0] 2> mtest:start(1). [<0.36.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 1 PROCESS(ES) IN 54.647 SECONDS 3> mtest:start(4). [<0.50.0>,<0.51.0>,<0.52.0>,<0.53.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 4 PROCESS(ES) IN 49.546 SECONDS I already have a speedup here, with only 4 processes. then 20 times: 2> mtest:start(20). [<0.270.0>,<0.271.0>,<0.272.0>,<0.273.0>,<0.274.0>, <0.275.0>,<0.276.0>,<0.277.0>,<0.278.0>,<0.279.0>,<0.280.0>, <0.281.0>,<0.282.0>,<0.283.0>,<0.284.0>,<0.285.0>,<0.286.0>, <0.287.0>,<0.288.0>,<0.289.0>] COMPUTED ADD TO 100000000 FOR 10 TIMES IN 20 PROCESS(ES) IN 188.824 SECONDS which averages to 9.44s per process. You can see that I get a 5.8 times speedup per process by running it that way. On the other hand, a big surprise is that it takes you 8 seconds for 1 process and 55 for me. I guess it could be that HiPE doesn't run on my windows version, with the benchmark being numeric computations by nature... 8> c(mtest, [native]). ./mtest.erl:none: Warning: this system is not configured for native-code compilation. On Sat, May 15, 2010 at 10:14 AM, Fred Hebert wrote: > Having 4 processes doesn't mean they'll all be on different cores. The VM > already runs maybe about 20-30 of them to begin with. > > You should see better concurrent behaviour if you were to do the test with > something like 50 processes on 4 cores versus 50 processes on a single one. > > There are other variables coming into play here, but in general, more > processes will make the behaviour better than fewer processes. The VM is > meant to run thousands and thousands of processes concurrently in larger > systems. Small benchmarks like this are likely to show weird results -- it's > not exactly what Erlang would be optimized for. > > On Sat, May 15, 2010 at 8:50 AM, Roberto Ostinelli wrote: > >> dear list, >> >> i'm performing some tests to optimize multicore usage. what i want to >> understand is the best way to use the 4 cores that my machine has, in >> a very simple erlang computation. >> >> i've written this module which basically spawns a very small number of >> processes which do a trivial but intensive computation, and report to >> a registered 'counter' process when done. >> >> here's the test module: >> >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> >> -module(mtest). >> -compile(export_all). >> -define(NUMCOMPUTE, 100000000). >> -define(NUMTIMES, 10). >> -define(TIMEOUT, 60000). >> >> start(ProcNum) -> >> % start counter >> register(counter, spawn(fun() -> counter_loop(now(), ProcNum, >> ProcNum) end)), >> % start compute processes >> [spawn(?MODULE, increase, [0, 0]) || _K <- lists:seq(1, ProcNum)]. >> >> % computational processes >> increase(?NUMCOMPUTE, ?NUMTIMES) -> counter ! finished; >> increase(?NUMCOMPUTE, Times) -> increase(0, Times + 1); >> increase(N, Times) -> increase(N + 1, Times). >> >> % counter loop >> counter_loop(Start, 0, ProcNum) -> >> T = timer:now_diff(now(), Start), >> io:format("COMPUTED ADD TO ~p FOR ~p TIMES IN ~p PROCESS(ES) IN ~p >> SECONDS~n", [?NUMCOMPUTE, ?NUMTIMES, ProcNum, T/1000000]); >> counter_loop(Start, ProcsLeft, ProcNum) -> >> receive >> finished -> >> counter_loop(Start, ProcsLeft - 1, ProcNum); >> _ -> >> counter_loop(Start, ProcsLeft, ProcNum) >> after ?TIMEOUT -> timeout >> end. >> >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> >> >> when i run this module with kernel poll and smp enabled, i get: >> >> >> roberto$ erl +K true >> Erlang R13B04 (erts-5.7.5) [source] [smp:4:4] [rq:4] [async-threads:0] >> [hipe] [kernel-poll:true] >> >> Eshell V5.7.5 (abort with ^G) >> 1> c(mtest). >> {ok,mtest} >> 2> mtest:start(1). >> [<0.40.0>] >> COMPUTED ADD TO 100000000 FOR 10 TIMES IN 1 PROCESS(ES) IN 8.100901 >> SECONDS >> 3> mtest:start(2). >> [<0.43.0>,<0.44.0>] >> COMPUTED ADD TO 100000000 FOR 10 TIMES IN 2 PROCESS(ES) IN 10.367364 >> SECONDS >> 4> mtest:start(3). >> [<0.47.0>,<0.48.0>,<0.49.0>] >> COMPUTED ADD TO 100000000 FOR 10 TIMES IN 3 PROCESS(ES) IN 13.541443 >> SECONDS >> 5> mtest:start(4). >> [<0.52.0>,<0.53.0>,<0.54.0>,<0.55.0>] >> COMPUTED ADD TO 100000000 FOR 10 TIMES IN 4 PROCESS(ES) IN 17.512607 >> SECONDS >> 6> >> >> >> i would have expected that, since 1 process running on 1 core takes >> around 8 seconds, running the code on 4 processes [hence on 4 cores] >> would have taken only marginally more time [for smp, startup, etc]. >> however, i see here that 4 cores are taking twice as much time that i >> would have expected. >> >> in this kind of situations, as reported also in other topics, it seems >> that using 4 erlang instances with smp disabled would definitely allow >> me to run the very same test in 8 seconds, not in 17. i'm prepared to >> loose a little for smp, which is normal, but adding twice up is >> definitely a high cost. >> >> i would have thought that this kind of parallelism would have been >> handled with no hassle in erlang: am i approaching this problem in a >> wrong manner? what should i do instead to achieve my expected results? >> >> thank you, >> >> r. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From jay@REDACTED Sat May 15 19:46:49 2010 From: jay@REDACTED (Jay Nelson) Date: Sat, 15 May 2010 10:46:49 -0700 Subject: Classification of concurrency patterns Message-ID: In an earlier thread I alluded to not understanding how to express concurrency patterns. My belief was that they are more complicated to categorize and differentiate than design patterns of OO because they cross the architecture rather than just a single software module. I wasn't sure what elements were necessary to describe them and think that is starting to show in the discussion on this thread. I think it will take a little analysis and negotiation to arrive at a reasonable description. Thus far, this thread has mainly enumerated specific instances of architectures without looking at the attributes and features of the entire problem space with a goal of identifying the measures which would differentiate approaches. A proper analysis should result in some as yet uncommon or undiscovered concurrency patterns by filling in missing examples on a diagram of possibilities. To make progress we need to look at the range of features needed to describe the domain of concurrency patterns. Here are some key attributes of a concurrent architecture that I can think of offhand, along with measures of each that could be used to classify them (considering only process-based concurrency), the main point being different ways of describing the processes that participate in a pattern: Number of processes - One => all OO design patterns fall in here - Few => typically task oriented processes with dedicated functionality - Dozens => typically partitioned systems with worker pools - Many => event driven spawning or grid-like computations - Dynamic => distributed hashes, communal participatory computation (SETI) Lifetime of processes - Static/Infinite => dedicated services essential to working system - Enduring => self-recovering stable functions (reconnect on disconnect), resource pool workers - Temporary => services needed on demand, caching - Ephemeral => reactive workers or task-oriented computation - Dormant => resource conserving (hibernation) Scheduling of processes - Once => static service - Periodic => cron-like services, Comet / AJAX polling - On Demand => event-driven services Static Organization of processes - Network computation topologies => hypercube, star, mesh, ring, etc. - Functional relationships => supervisors, serial servers, pub/ sub, pool - Partitioning => isolation of failure, allocation of resources Dynamic Organization of processes - Load adaptive => efficient resource allocating services - Demand driven => swarm computing - Migratory => mobile agents, resource discovery, continuation patterns - Coordinated => failover / takeover, command and control, leader election - Participatory => distributed hash, SETI Data flow through processes - Static => networked dataflow, traditional partitioned - Affinity-based => data flocking for efficient computation - Routed / Queue distributed => pub/sub, worker pools - Activation propagation => traditional dataflow, pipeline - Central distributed => serial server, rendezvous - Scatter / Gather => map reduce, grid - Network overlay => SIMD grid Location of processes - Static => traditional concurrency - Dynamic => mobile agents, adaptive load systems - Admin directed => command and control concurrency - Resource associated => adaptive load, efficient data computation Process-Mapped Resource Access - Continuous => traditional - Repeated static => pool workers - Repeated dynamic => event driven, load adaptive - One-shot => caching, dataflow, tcp proxy As you can see, the OO patterns eliminate a few dimensions of the range of possibilities and therefore are presumably easier to describe. Not all of these features are necessary simultaneously to classify the entire domain (as seen by some of my overlapping descriptions), arguments exist for selective sets of them or combinations. I think that because a concurrent system is more complex and dynamic, it is necessary to describe an architecture with both static attributes (typically called "the architecture") and dynamic attributes (process lifecycle, dataflow) over time. I think this is a highly relevant thread, and an important topic for the Concurrency Oriented Programming Language (COPL) community to come to agreement on. Doing so will advance software and training at the pace of the multi-core options that are becoming available. I think this outline could form the basis of a collaborative wiki to discover a better classification hierarchy. jay From torben.lehoff@REDACTED Sat May 15 22:38:32 2010 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Sat, 15 May 2010 22:38:32 +0200 Subject: [erlang-questions] wise binary tails? In-Reply-To: <4BEAEBBE.1030405@eonblast.com> References: <4BEAEBBE.1030405@eonblast.com> Message-ID: I think you should read http://erlang.org/doc/efficiency_guide/binaryhandling.html and try to see if it can explain your question - it seems that your code is what is the most optimal, but I would recommend reading it rather than trusting my quick gaze over the documentation. Cheers, Torben On Wed, May 12, 2010 at 19:56, Henning Diedrich wrote: > If I have code like this: > > traverse(<>) -> [ Bar | traverse(Rest) ]; > > traverse(<<>>) -> []. > > traverse(FooBin). > > - or - > > [ Bar || <> <= FooBin ]. > > is Erlang internally doing the wise thing of simply shifting a pointer into > the low level buffer where FooBin is stored. > > And thus avoids copying the rest of the buffer again and again? > > Thanks! > Henning > -- http://www.linkedin.com/in/torbenhoffmann From fritchie@REDACTED Sat May 15 23:08:51 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sat, 15 May 2010 16:08:51 -0500 Subject: net_kernel hang, perhaps blocked by busy_dist_port race? Message-ID: <64148.1273957731@snookles.snookles.com> Hi, all. We've been bitten by a rather mysterious bug that has disrupted Erlang message passing on roughly 10% of all nodes in a 100+ node cluster. The same thing happened on 10 nodes within a 2-3 second time window. No further communication with the affected nodes via Erlang message passing is possible. For details, see a post by the same subject to the erlang-bugs list. R13B04 on x86-64 Linux boxes. Steps #1-7 did indeed happen in that order, there's little doubt, thanks to some chatty app logging. -Scott P.S. For those of you still interested, here's the intro to the erlang-bugs posting. I'm wondering if there's a possible race condition when two nodes A and Z are communicating with each other, like this: 1. Z makes a bunch of RPCs to A. 2. A starts sending RPC replies to Z. 3. Z decides to behave erratically, cause unknown. 4. A's TCP connection to Z becomes "busy", probably because Z cannot or will not read data on the A <-> Z TCP connection. 5. All processes on A that are trying to reply to Z are blocked and unscheduled; 'busy_dist_port' messages are generated for all of them. 6. The 'net_kernel' process on A is one of the procs blocked by the 'busy_dist_port' events. 7. A's connection to Z is broken. The system message reported is: [{nodedown_reason,connection_closed},{node_type,visible}] ... and then A's 'net_kernel' process remains blocked forever? Or is alive but isn't working correctly? "erl -sname tmp$$ -remsh app@REDACTED" will fail, for example. From luismarianoguerra@REDACTED Sat May 15 23:55:19 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Sat, 15 May 2010 18:55:19 -0300 Subject: [ANN][efene] tutorial part 1 draft finished, need early reviewers Message-ID: I started working on an efene tutorial that starts with a hello world and ends with a fully usable web application. The tutorial is divided in 4 parts, right now the first part is finished, but needs some review from people actually following the tutorial. The tutorial is located here: http://www.marianoguerra.com.ar/efene/tutorial/ http://www.marianoguerra.com.ar/efene/tutorial/part1.html every section of the tutorial has a comment section, you can comment about anything you think about that section, from errors, clarifications, or asking for some modification or addition. I will continue with part 2 next week, that part will start creating a real web application. Any question and/or observation is really useful. thanks. From raould@REDACTED Sun May 16 10:43:27 2010 From: raould@REDACTED (Raoul Duke) Date: Sun, 16 May 2010 01:43:27 -0700 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Sat, May 15, 2010 at 10:46 AM, Jay Nelson wrote: > To make progress we need to look at the range of features needed to describe > the domain of concurrency patterns. + a bazillion. in other words, i sometimes think that if you can't draw it up as a matrix/grid that shows the possible options, you don't understand the space well enough. it is especially useful/important because - as one of the Sutherland brothers said - once you can render it thus, you can see which parts of the grid are empty, and ponder why. sincerely. From hd2010@REDACTED Mon May 17 00:04:28 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 17 May 2010 00:04:28 +0200 Subject: Defensive Programming Message-ID: <4BF06BEC.1070106@eonblast.com> Hi list, I have a question about the Erlang way. I realize I keep programming defensive using exceptions/error calls to *label failure* more precisely. E.g. ( - PRESUMABLY WRONG - ) login(Name, Password) -> ... case gen_tcp:send(Socket, term_to_binary(Str)) of ok -> ok; _ -> erlang:error(*sending_failed, {Str }*) end ... Instead of simply login(Name, Password) -> ... ok = gen_tcp:send(Socket, term_to_binary(Str)), ... It feels wrong in Erlang and I wonder if I am missing out on something. If somebody has a pointer to read up on this, thanks in advance, Henning From steven.charles.davis@REDACTED Mon May 17 00:42:07 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 16 May 2010 15:42:07 -0700 (PDT) Subject: Defensive Programming In-Reply-To: <4BF06BEC.1070106@eonblast.com> References: <4BF06BEC.1070106@eonblast.com> Message-ID: <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> Hi Henning, This isn't intended to be any kind of definite answer, and I'm interested to see how others respond. In my case how I deal with this question is that I'd probably ask myself: *Is this exceptional case _recoverable_,? e.g. in the case of the tcp send, you could try to resend the data, so the first idiom may do it. If it's not recoverable, I'd just let it crash out. In fact I would likely force it to crash out using the second idiom. If I want to get fancy with error handling, I find that try..catch at the top level is usually enough. A further aside to this is that a certain Mr Armstrong once introduced me to an interesting "transactional" idiom also, namely: try begin ... end catch ... Regards, /s On May 16, 5:04?pm, Henning Diedrich wrote: > Hi list, > > I have a question about the Erlang way. > > I realize I keep programming defensive using exceptions/error calls to > *label failure* more precisely. > > E.g. ( - PRESUMABLY WRONG - ) > > login(Name, Password) -> > > ? ? ? ?... > ? ? ? ? case > ? ? ? ? ? ? ? ? gen_tcp:send(Socket, term_to_binary(Str)) of > ? ? ? ? ? ? ? ? ? ? ?ok -> ok; > ? ? ? ? ? ? ? ? ? ? ?_ -> erlang:error(*sending_failed, {Str }*) > ? ? ? ?end > ? ? ? ?... > > Instead of simply > > login(Name, Password) -> > > ? ? ? ?... > ? ? ? ? ok = gen_tcp:send(Socket, term_to_binary(Str)), > ? ? ? ?... > > It feels wrong in Erlang and I wonder if I am missing out on something. > > If somebody has a pointer to read up on this, thanks in advance, > > Henning > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From hd2010@REDACTED Mon May 17 01:12:15 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 17 May 2010 01:12:15 +0200 Subject: [erlang-questions] Re: Defensive Programming In-Reply-To: <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> Message-ID: <4BF07BCF.7020309@eonblast.com> Thanks Steve, that question of "is it recoverable?" makes a lot of sense to clarify. I am taking note. In my case I am writing a lib function for others to use, which is exactly why I would like to give them more information on the error. Even though they might not be able to recover it, they should at least know why it didn't work, instead of just getting a bad match for "ok = " I meanwhile conclude that the trimmest variant will be to care less and demand that the user of the function test for the return value directly handed over from the gen_tcp:send() call. Making it login(Name, Password) -> ... gen_tcp:send(Socket, term_to_binary(Str)). That looks the Erlangest an hands responsibility to the person who can actually answer the 'recoverability question', the user of the lib. I realize I'd like to take away chances of frustration, puffing it up in the process. Thanks for taking a look! Henning Steve Davis wrote: > Hi Henning, > > This isn't intended to be any kind of definite answer, and I'm > interested to see how others respond. > > In my case how I deal with this question is that I'd probably ask > myself: > > *Is this exceptional case _recoverable_,? e.g. in the case of the tcp > send, you could try to resend the data, so the first idiom may do it. > > If it's not recoverable, I'd just let it crash out. In fact I would > likely force it to crash out using the second idiom. > > If I want to get fancy with error handling, I find that try..catch at > the top level is usually enough. > > A further aside to this is that a certain Mr Armstrong once introduced > me to an interesting "transactional" idiom also, namely: > > try begin > ... > end catch > ... > > Regards, > > /s > > On May 16, 5:04 pm, Henning Diedrich wrote: > >> Hi list, >> >> I have a question about the Erlang way. >> >> I realize I keep programming defensive using exceptions/error calls to >> *label failure* more precisely. >> >> E.g. ( - PRESUMABLY WRONG - ) >> >> login(Name, Password) -> >> >> ... >> case >> gen_tcp:send(Socket, term_to_binary(Str)) of >> ok -> ok; >> _ -> erlang:error(*sending_failed, {Str }*) >> end >> ... >> >> Instead of simply >> >> login(Name, Password) -> >> >> ... >> ok = gen_tcp:send(Socket, term_to_binary(Str)), >> ... >> >> It feels wrong in Erlang and I wonder if I am missing out on something. >> >> If somebody has a pointer to read up on this, thanks in advance, >> >> Henning >> >> -- >> You received this message because you are subscribed to the Google Groups "Erlang Programming" group. >> To post to this group, send email to erlang-programming@REDACTED >> To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED >> For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From ok@REDACTED Mon May 17 04:55:44 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 17 May 2010 14:55:44 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> Message-ID: <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> On May 15, 2010, at 7:05 AM, Eric Newhuis (personal) wrote: > Consider: > > X = case Any of > very_long_pattern_or_whatever -> > very_long_pattern_or_whatever > end. > > > I'd rather abbreviate that like this. > > X = case Any of > very_long_pattern_or_whatever -> _ > end. > > It also seems more readable to me than: > > X = case Any of > very_long_pattern_or_whatever=Y -> Y > end. Abusing the wild-card like that would be very confusing. It certainly is NOT more readable than X = case Any of Y = very_long_pattern_or_whatever -> Y end or X = case Y = Any of very_long_pattern_or_whatever -> Y end or even Y = Any, X = case Y of very_long_pattern_or_whatever -> Y end We already have so many readable ways to do it that we're much better off not adding an unreadable one. From mazen.harake@REDACTED Mon May 17 08:48:48 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 17 May 2010 09:48:48 +0300 Subject: [erlang-questions] Re: Defensive Programming In-Reply-To: <4BF07BCF.7020309@eonblast.com> References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> <4BF07BCF.7020309@eonblast.com> Message-ID: <4BF0E6D0.4010609@erlang-solutions.com> I wrote this once... see if it makes sense: http://mazenharake.wordpress.com/2009/09/14/let-it-crash-the-right-way/ /Mazen On 17/05/2010 02:12, Henning Diedrich wrote: > Thanks Steve, > > that question of "is it recoverable?" makes a lot of sense to clarify. > I am taking note. > > In my case I am writing a lib function for others to use, which is > exactly why I would like to give them more information on the error. > > Even though they might not be able to recover it, they should at least > know why it didn't work, instead of just getting a bad match for "ok = " > > I meanwhile conclude that the trimmest variant will be to care less > and demand that the user of the function test for the return value > directly handed over from the gen_tcp:send() call. Making it > > login(Name, Password) -> > > ... > gen_tcp:send(Socket, term_to_binary(Str)). > > > > That looks the Erlangest an hands responsibility to the person who can > actually answer the 'recoverability question', the user of the lib. > > I realize I'd like to take away chances of frustration, puffing it up > in the process. > > Thanks for taking a look! > Henning > > > > Steve Davis wrote: >> Hi Henning, >> >> This isn't intended to be any kind of definite answer, and I'm >> interested to see how others respond. >> >> In my case how I deal with this question is that I'd probably ask >> myself: >> >> *Is this exceptional case _recoverable_,? e.g. in the case of the tcp >> send, you could try to resend the data, so the first idiom may do it. >> >> If it's not recoverable, I'd just let it crash out. In fact I would >> likely force it to crash out using the second idiom. >> >> If I want to get fancy with error handling, I find that try..catch at >> the top level is usually enough. >> >> A further aside to this is that a certain Mr Armstrong once introduced >> me to an interesting "transactional" idiom also, namely: >> >> try begin >> ... >> end catch >> ... >> >> Regards, >> >> /s >> >> On May 16, 5:04 pm, Henning Diedrich wrote: >>> Hi list, >>> >>> I have a question about the Erlang way. >>> >>> I realize I keep programming defensive using exceptions/error calls to >>> *label failure* more precisely. >>> >>> E.g. ( - PRESUMABLY WRONG - ) >>> >>> login(Name, Password) -> >>> >>> ... >>> case >>> gen_tcp:send(Socket, term_to_binary(Str)) of >>> ok -> ok; >>> _ -> erlang:error(*sending_failed, {Str }*) >>> end >>> ... >>> >>> Instead of simply >>> >>> login(Name, Password) -> >>> >>> ... >>> ok = gen_tcp:send(Socket, term_to_binary(Str)), >>> ... >>> >>> It feels wrong in Erlang and I wonder if I am missing out on something. >>> >>> If somebody has a pointer to read up on this, thanks in advance, >>> >>> Henning >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Erlang Programming" group. >>> To post to this group, send email to >>> erlang-programming@REDACTED >>> To unsubscribe from this group, send email to >>> erlang-programming+unsubscribe@REDACTED >>> For more options, visit this group >>> athttp://groups.google.com/group/erlang-programming?hl=en. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From enewhuis@REDACTED Mon May 17 16:00:56 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Mon, 17 May 2010 09:00:56 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> Message-ID: <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> For the record, I might still disagree, so far. I'm not sure. Simply for argument's sake... The context in which this might be maximally useful is the following. case some_module:some_function(...) of {some, pattern} -> _; {some, other, pattern} -> _; _ -> whatever end. Note that I belong to the school of philosophy that suggests that the number of temporary variables should be minimized. I don't understand why the above would be called wild-card abuse. It is clear from context that the wild-card represents something other than matching. We've seen this idea before. There are those grammars that expose variables whose value is whatever matched. I suppose the following is where this great idea of mine might break down. case some_module:some_function(...) of {some, pattern} -> {encapsulated, _}; {some, other, pattern} -> {another, _, encapsulation} end. Although I still don't have a problem with that. From context I know that the right hand side of the arrow isn't pattern matching. I guess where readability might break down is in nesting: case some_module:some_function(...) of {some, _, pattern} -> % _1 case _ of -> % _2 {some, great, pattern} -> not_so_bad; _ -> % _3 {_, Kind, _} = _, % _4, _5, _6 Kind end end. Although I can still read the above once I learn that underscore ('_') is context sensitive. _1 :: any() _2 :: {some, any(), pattern} _3 :: {some, any(), pattern}, not {some, great, pattern} _4 :: some _5 :: pattern _6 :: _3 On May 16, 2010, at 9:55 PM, Richard O'Keefe wrote: > > On May 15, 2010, at 7:05 AM, Eric Newhuis (personal) wrote: > >> Consider: >> >> X = case Any of >> very_long_pattern_or_whatever -> >> very_long_pattern_or_whatever >> end. >> >> >> I'd rather abbreviate that like this. >> >> X = case Any of >> very_long_pattern_or_whatever -> _ >> end. >> >> It also seems more readable to me than: >> >> X = case Any of >> very_long_pattern_or_whatever=Y -> Y >> end. > > Abusing the wild-card like that would be very confusing. > It certainly is NOT more readable than > > X = case Any > of Y = very_long_pattern_or_whatever -> > Y > end > or > X = case Y = Any > of very_long_pattern_or_whatever -> Y > end > > or even > Y = Any, > X = case Y > of very_long_pattern_or_whatever -> Y > end > > We already have so many readable ways to do it that > we're much better off not adding an unreadable one. > > From rvirding@REDACTED Mon May 17 16:36:54 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 17 May 2010 16:36:54 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> Message-ID: On 17 May 2010 16:00, Eric Newhuis (personal) wrote: > ... > > I guess where readability might break down is in nesting: > > case some_module:some_function(...) of > ? ? ? ?{some, _, pattern} -> % _1 > ? ? ? ? ? ? ? ?case _ of -> ?% _2 > ? ? ? ? ? ? ? ? ? ? ? ?{some, great, pattern} -> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?not_so_bad; > ? ? ? ? ? ? ? ? ? ? ? ?_ -> % _3 > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{_, Kind, _} = _, % _4, _5, _6 > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Kind > ? ? ? ? ? ? ? ?end > end. > > Although I can still read the above once I learn that underscore ('_') is context sensitive. > > _1 :: any() > _2 :: {some, any(), pattern} > _3 :: {some, any(), pattern}, not {some, great, pattern} > _4 :: some > _5 :: pattern > _6 :: _3 That is completely "unworkable". To put it very mildly. To have '_' in a pattern sometimes be a don't care which matches anything and is dropped and sometimes be a value which is matched against is just not acceptable. To make it even worse it changes its value in the body which would be the only case where this happens. Also the usage in your 4,5,6 case is inconsistent with the other examples as Pat = Expr is really just a shorter form of: case Expr of Pat -> end and completely equivalent to it except for the error message. So you really have case _ of {_,Kind,_} -> Kind end which doesn't behave as you imply. Sorry no! Robert From silent_vendetta@REDACTED Mon May 17 16:58:10 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 17 May 2010 07:58:10 -0700 Subject: Mnesia questions Message-ID: I've looked all over to get the answers to these questions but I can't seem to find the answers I'm looking for, so here goes: Mnesia disk_copy tables to NOT use dets, but use a type of log file to handle the disk part of the storage. Does 32 bit Erlang still have a size limit with these files? What about 64 bit? The file size limit is per table anyway and I shouldn't have a problem with disc_copies if I make sure to break up my data set accordingly, correct? I keep reading there there will be a performance hit way before you reach the file size limit with dets anyway, does that hold true for disc_copies as well? Anyone know at what table size the performance begins to degrade significantly? I think those will do to start with. Any answers would be greatly appreciated. _________________________________________________________________ The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 From enewhuis@REDACTED Mon May 17 17:21:04 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Mon, 17 May 2010 10:21:04 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> Message-ID: I am NOT proposing that the wild-card character behave selectively in pattern matching. That just isn't part of what I am proposing. As far as I can tell, the wild-card is not presently allowed at all in the context that I have proposed. So there is no formal ambiguity as far as I can tell. And it wouldn't break any existing code. ...unless I am missing something basic. I am not proposing that wild-card ('_') ever be reinterpreted on the left hand side of an arrow ('->'). To make this clearer one could introduce a special operator placeholder ('@'). >> case some_module:some_function(...) of >> {some, _, pattern} -> % _1 >> case @ of -> % _2 >> {some, great, pattern} -> >> not_so_bad; >> _ -> % _3 >> {_, Kind, _} = @, % _4, _5, _6 >> Kind >> end >> end. On May 17, 2010, at 9:36 AM, Robert Virding wrote: > On 17 May 2010 16:00, Eric Newhuis (personal) wrote: >> ... >> >> I guess where readability might break down is in nesting: >> >> case some_module:some_function(...) of >> {some, _, pattern} -> % _1 >> case _ of -> % _2 >> {some, great, pattern} -> >> not_so_bad; >> _ -> % _3 >> {_, Kind, _} = _, % _4, _5, _6 >> Kind >> end >> end. >> >> Although I can still read the above once I learn that underscore ('_') is context sensitive. >> >> _1 :: any() >> _2 :: {some, any(), pattern} >> _3 :: {some, any(), pattern}, not {some, great, pattern} >> _4 :: some >> _5 :: pattern >> _6 :: _3 > > That is completely "unworkable". To put it very mildly. To have '_' in > a pattern sometimes be a don't care which matches anything and is > dropped and sometimes be a value which is matched against is just not > acceptable. To make it even worse it changes its value in the body > which would be the only case where this happens. Also the usage in > your 4,5,6 case is inconsistent with the other examples as Pat = Expr > is really just a shorter form of: > > case Expr of > Pat -> > end > > and completely equivalent to it except for the error message. So you really have > > case _ of > {_,Kind,_} -> Kind > end > > which doesn't behave as you imply. > > Sorry no! > > Robert From egil@REDACTED Mon May 17 17:35:29 2010 From: egil@REDACTED (=?UTF-8?B?QmrDtnJuLUVnaWwgRGFobGJlcmc=?=) Date: Mon, 17 May 2010 17:35:29 +0200 Subject: [erlang-questions] low overhead profiler for erlang? In-Reply-To: <4BDEEA84.2030101@gmail.com> References: <4BDEEA84.2030101@gmail.com> Message-ID: <4BF16241.40001@erix.ericsson.se> There is improvement on the way for eprof (and scaling improvements on cprof). There is a dev branch for on github for new breakpoints and an improved eprof. Anyone can try it out. Note that this branch will get rebased and revision will be made but is should be stable enough for testing. Some features are just stubs at the moment. git fetch git://github.com/psyeugenic/otp.git wip/call_time_trace_clean Regards, Bj?rn-Egil Erlang/OTP On 2010-05-03 17:23, mabrek wrote: > Hello. > > Is there a way to do profiling with low overhead? > > I tried eprof and fprof, in both cases profiling slowed down my application 10 times. It was able to serve 40 http requests per second without profiling but when I enabled profiling speed dropped down to 4 requests per second. > It seems that such a big overhead skews results of profiling adding time to frequently called functions. > I knew that the most time consuming part of my application was database queries (I proved that later by replacing external SQL database with mnesia RAM table), but profiler failed to show it. Frequently called xmerl xml escaping functions were above all in a sorted by accumulated time results. > > Regards, > Anton Lebedevich. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From rtrlists@REDACTED Mon May 17 18:09:07 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 17 May 2010 17:09:07 +0100 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> Message-ID: What's wrong with these? So, your temp var is now called X instead of _ and you always know what you're looking at. On Mon, May 17, 2010 at 3:00 PM, Eric Newhuis (personal) wrote: > > case some_module:some_function(...) of > {some, pattern} -> _; > {some, other, pattern} -> _; > _ -> whatever > end. > > case X = some_module:some_function(...) of {some, pattern} -> X; {some, other, pattern} -> X; _ -> whatever end. case some_module:some_function(...) of > {some, pattern} -> > {encapsulated, _}; > {some, other, pattern} -> > {another, _, encapsulation} > end. > > case X = some_module:some_function(...) of {some, pattern} -> {encapsulated, X}; {some, other, pattern} -> {another, X, encapsulation} end. case some_module:some_function(...) of > {some, _, pattern} -> % _1 > case _ of -> % _2 > {some, great, pattern} -> > not_so_bad; > _ -> % _3 > {_, Kind, _} = _, % _4, _5, _6 > Kind > end > end. > > case some_module:some_function(...) of {some, X, pattern} -> % _1 case X of -> % _2 {some, great, pattern} -> not_so_bad; _ -> % _3 {_, Kind, _} = X, % _4, _5, _6 Kind end end. Robby From ulf.wiger@REDACTED Mon May 17 18:14:29 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 17 May 2010 18:14:29 +0200 Subject: [erlang-questions] Mnesia questions In-Reply-To: References: Message-ID: <4BF16B65.9090607@erlang-solutions.com> Chris Hicks wrote: > I've looked all over to get the answers to these questions but I > can't seem to find the answers I'm looking for, so here goes: Mnesia > disk_copy tables to NOT use dets, but use a type of log file to > handle the disk part of the storage. Does 32 bit Erlang still have a > size limit with these files? Disk log files are only limited by the 64-bit relative addressing. > What about 64 bit? The file size limit It's the same, as far as I know. This goes for dets too. The main difference is that disk_copies are also kept in RAM, and this capacity is of course increased significantly. :) > is per table anyway and I shouldn't have a problem with disc_copies > if I make sure to break up my data set accordingly, correct? I keep > reading there there will be a performance hit way before you reach > the file size limit with dets anyway, does that hold true for > disc_copies as well? Disc_copies use the disk competely differently from dets. They rely heavily on append, and basically streaming data to and from disk. Disks generally do fairly well with this kind of access pattern. > Anyone know at what table size the performance > begins to degrade significantly? It's a bit hardware-specific, but essentially, performance falls off a cliff when your data set gets poor cache locality. When this happens depends on your OS, disk subsystem etc., but once you become limited by disk seek time, the performance is reasonably predictable - roughly speaking, around 20 writes/sec. SSD drives are much less cache-dependent, but I have no personal experience using dets on SSD drives. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From g9414002.pccu.edu.tw@REDACTED Mon May 17 18:51:05 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 18 May 2010 00:51:05 +0800 Subject: Does Erlang Treat Everything Function? Message-ID: Hi, Erlang-lovers! I'm writing Erlang tutorials in my local language, Chinese. I felt in thinking about a word "Everything is functions." Could I say that Erlang realizes the word too? That is, to say even if a term is number, it's also a function: for example, 0 is a function 0/0. Is it right? Best Regards, YHH. From mazen.harake@REDACTED Mon May 17 19:06:09 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 17 May 2010 20:06:09 +0300 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: References: Message-ID: <4BF17781.4080102@erlang-solutions.com> No this is not true. See here for functions: http://www.erlang.org/doc/reference_manual/functions.html See here for higher order functions: http://www.erlang.org/doc/reference_manual/data_types.html The rest are just data types. A Process is an instance of execution inside a given function. Hope this clears it up a little. Good luck, /Mazen On 17/05/2010 19:51, ??? (Yau-Hsien Huang) wrote: > Hi, Erlang-lovers! I'm writing Erlang tutorials in my local > language, Chinese. I felt in thinking about a word > "Everything is functions." > Could I say that Erlang realizes the word too? > That is, to say even if a term is number, it's also a function: > for example, 0 is a function 0/0. Is it right? > > Best Regards, > YHH. > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From hd2010@REDACTED Mon May 17 19:46:54 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 17 May 2010 19:46:54 +0200 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: References: Message-ID: <4BF1810E.3040906@eonblast.com> :-) How is that written in Chinese? I'd like to use that as a motto. Thanks, Henning ??? (Yau-Hsien Huang) wrote: > Hi, Erlang-lovers! I'm writing Erlang tutorials in my local > language, Chinese. I felt in thinking about a word > "Everything is functions." > Could I say that Erlang realizes the word too? > That is, to say even if a term is number, it's also a function: > for example, 0 is a function 0/0. Is it right? > > Best Regards, > YHH. > > From garret.smith@REDACTED Mon May 17 20:15:52 2010 From: garret.smith@REDACTED (Garret Smith) Date: Mon, 17 May 2010 11:15:52 -0700 Subject: log_mf_h and rb do not handle terms over 65k Message-ID: I was logging a couple very large terms using error_logger:info_report and log_mf_h during application start, and was always unable to read most of the first log file generated. This was mystifying me for a while until I finally traced the source. log_mf_h and rb use a 2-byte length indicator for the term they are about to read/write, therefore they cannot handle terms over 65,536 bytes in length. See handle_event/2 in log_mf_h and read_report/1 in rb Increasing the length field would break backwards compatibility, but maybe it would be prudent to truncate the term to the max length or replace it with a "term to big" message so that the rest of the log file is not corrupted? When this happens in my application, the rest of the log file is unreadable. Has anyone else experienced this or know of a workaround (other than the obvious of checking the term size in my application code)? Thanks, Garret Smith From g9414002.pccu.edu.tw@REDACTED Mon May 17 22:06:09 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 18 May 2010 04:06:09 +0800 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: <4BF1810E.3040906@eonblast.com> References: <4BF1810E.3040906@eonblast.com> Message-ID: Really? Even if it's not actually true that Erlang treats everything function? OK. I tried writing that ???????? "Everything is function." :D On Tue, May 18, 2010 at 1:46 AM, Henning Diedrich wrote: > :-) How is that written in Chinese? I'd like to use that as a motto. > > Thanks, > Henning > > > ??? (Yau-Hsien Huang) wrote: > >> Hi, Erlang-lovers! I'm writing Erlang tutorials in my local >> language, Chinese. I felt in thinking about a word >> "Everything is functions." >> Could I say that Erlang realizes the word too? >> That is, to say even if a term is number, it's also a function: >> for example, 0 is a function 0/0. Is it right? >> >> Best Regards, >> YHH. >> >> >> > From rvirding@REDACTED Mon May 17 22:08:32 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 17 May 2010 22:08:32 +0200 Subject: [erlang-questions] Re: Defensive Programming In-Reply-To: <4BF0E6D0.4010609@erlang-solutions.com> References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> <4BF07BCF.7020309@eonblast.com> <4BF0E6D0.4010609@erlang-solutions.com> Message-ID: I was going to say that you should handle the error where it is most sensible to do so but you said it better. :-) Robert 2010/5/17 Mazen Harake : > I wrote this once... see if it makes sense: > http://mazenharake.wordpress.com/2009/09/14/let-it-crash-the-right-way/ > > /Mazen > > > On 17/05/2010 02:12, Henning Diedrich wrote: >> >> Thanks Steve, >> >> that question of "is it recoverable?" makes a lot of sense to clarify. I >> am taking note. >> >> In my case I am writing a lib function for others to use, which is exactly >> why I would like to give them more information on the error. >> >> Even though they might not be able to recover it, they should at least >> know why it didn't work, instead of just getting a bad match for "ok = " >> >> I meanwhile conclude that the trimmest variant will be to care less and >> demand that the user of the function test for the return value directly >> handed over from the gen_tcp:send() call. Making it >> >> login(Name, Password) -> >> >> ? ? ?... >> ? ? ? gen_tcp:send(Socket, term_to_binary(Str)). >> >> >> >> That looks the Erlangest an hands responsibility to the person who can >> actually answer the 'recoverability question', the user of the lib. >> >> I realize I'd like to take away chances of frustration, puffing it up in >> the process. >> >> Thanks for taking a look! >> Henning >> >> >> >> Steve Davis wrote: >>> >>> Hi Henning, >>> >>> This isn't intended to be any kind of definite answer, and I'm >>> interested to see how others respond. >>> >>> In my case how I deal with this question is that I'd probably ask >>> myself: >>> >>> *Is this exceptional case _recoverable_,? e.g. in the case of the tcp >>> send, you could try to resend the data, so the first idiom may do it. >>> >>> If it's not recoverable, I'd just let it crash out. In fact I would >>> likely force it to crash out using the second idiom. >>> >>> If I want to get fancy with error handling, I find that try..catch at >>> the top level is usually enough. >>> >>> A further aside to this is that a certain Mr Armstrong once introduced >>> me to an interesting "transactional" idiom also, namely: >>> >>> try begin >>> ... >>> end catch >>> ... >>> >>> Regards, >>> >>> /s >>> >>> On May 16, 5:04 pm, Henning Diedrich wrote: >>>> >>>> Hi list, >>>> >>>> I have a question about the Erlang way. >>>> >>>> I realize I keep programming defensive using exceptions/error calls to >>>> *label failure* more precisely. >>>> >>>> E.g. ( - PRESUMABLY WRONG - ) >>>> >>>> login(Name, Password) -> >>>> >>>> ? ? ? ... >>>> ? ? ? ?case >>>> ? ? ? ? ? ? ? ?gen_tcp:send(Socket, term_to_binary(Str)) of >>>> ? ? ? ? ? ? ? ? ? ? ok -> ok; >>>> ? ? ? ? ? ? ? ? ? ? _ -> erlang:error(*sending_failed, {Str }*) >>>> ? ? ? end >>>> ? ? ? ... >>>> >>>> Instead of simply >>>> >>>> login(Name, Password) -> >>>> >>>> ? ? ? ... >>>> ? ? ? ?ok = gen_tcp:send(Socket, term_to_binary(Str)), >>>> ? ? ? ... >>>> >>>> It feels wrong in Erlang and I wonder if I am missing out on something. >>>> >>>> If somebody has a pointer to read up on this, thanks in advance, >>>> >>>> Henning >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Erlang Programming" group. >>>> To post to this group, send email to >>>> erlang-programming@REDACTED >>>> To unsubscribe from this group, send email to >>>> erlang-programming+unsubscribe@REDACTED >>>> For more options, visit this group >>>> athttp://groups.google.com/group/erlang-programming?hl=en. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From g9414002.pccu.edu.tw@REDACTED Mon May 17 22:23:13 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 18 May 2010 04:23:13 +0800 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: <4BF17781.4080102@erlang-solutions.com> References: <4BF17781.4080102@erlang-solutions.com> Message-ID: Thank you, Mazen. Part of the functional programming concept of mine is from Haskell and some readings more general, so when I heard that Erlang is functional Haskell and other things popped out on my brain. YHH 2010/5/18 Mazen Harake > No this is not true. > > See here for functions: > http://www.erlang.org/doc/reference_manual/functions.html > See here for higher order functions: > http://www.erlang.org/doc/reference_manual/data_types.html > > The rest are just data types. > > A Process is an instance of execution inside a given function. > > Hope this clears it up a little. > > Good luck, > > /Mazen > > > On 17/05/2010 19:51, ??? (Yau-Hsien Huang) wrote: > >> Hi, Erlang-lovers! I'm writing Erlang tutorials in my local >> language, Chinese. I felt in thinking about a word >> "Everything is functions." >> Could I say that Erlang realizes the word too? >> That is, to say even if a term is number, it's also a function: >> for example, 0 is a function 0/0. Is it right? >> >> Best Regards, >> YHH. >> >> >> > > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become > ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > From hd2010@REDACTED Mon May 17 22:29:55 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 17 May 2010 22:29:55 +0200 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: References: <4BF1810E.3040906@eonblast.com> Message-ID: <4BF1A743.6040407@eonblast.com> Cool! Thanks! Yes, even though it may not be true. Bill would say it depends on your definition of "function", right. (Rgh. Can't paste it into Erlang ISO Latin-1 source :-Darn.) Henning ??? (Yau-Hsien Huang) wrote: > Really? Even if it's not actually true that Erlang treats everything > function? > > OK. I tried writing that > > ???????? > > "Everything is function." > > :D > > > On Tue, May 18, 2010 at 1:46 AM, Henning Diedrich wrote: > > >> :-) How is that written in Chinese? I'd like to use that as a motto. >> >> Thanks, >> Henning >> >> >> ??? (Yau-Hsien Huang) wrote: >> >> >>> Hi, Erlang-lovers! I'm writing Erlang tutorials in my local >>> language, Chinese. I felt in thinking about a word >>> "Everything is functions." >>> Could I say that Erlang realizes the word too? >>> That is, to say even if a term is number, it's also a function: >>> for example, 0 is a function 0/0. Is it right? >>> >>> Best Regards, >>> YHH. >>> >>> >>> >>> > > -- *Henning Diedrich* CEO Eonblast Corporation hdiedrich@REDACTED +1.404.418.5002 w www.eonblast.com This email contains confidential and/or privileged information. If you are not the intended recipient (or have received this email in error) please notify the sender immediately and destroy this email. Any unauthorized copying, disclosure or distribution of the material in this email is prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 4856 bytes Desc: not available URL: From g9414002.pccu.edu.tw@REDACTED Mon May 17 22:37:50 2010 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 18 May 2010 04:37:50 +0800 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: <4BF1A743.6040407@eonblast.com> References: <4BF1810E.3040906@eonblast.com> <4BF1A743.6040407@eonblast.com> Message-ID: Oh...... Two code pages 950 (Big5) and 65001 (UTF-8) support Chinese Traditional words. Try to use another file code page which is one of these two, and the word can be pasted. Best Regards. On Tue, May 18, 2010 at 4:29 AM, Henning Diedrich wrote: > Cool! Thanks! Yes, even though it may not be true. Bill would say it > depends on your definition of "function", right. > > (Rgh. Can't paste it into Erlang ISO Latin-1 source :-Darn.) > > Henning > > ??? (Yau-Hsien Huang) wrote: > > Really? Even if it's not actually true that Erlang treats everything > function? > > OK. I tried writing that > > ???????? > > "Everything is function." > > :D > > > On Tue, May 18, 2010 at 1:46 AM, Henning Diedrich wrote: > > > > :-) How is that written in Chinese? I'd like to use that as a motto. > > Thanks, > Henning > > > ??? (Yau-Hsien Huang) wrote: > > > > Hi, Erlang-lovers! I'm writing Erlang tutorials in my local > language, Chinese. I felt in thinking about a word > "Everything is functions." > Could I say that Erlang realizes the word too? > That is, to say even if a term is number, it's also a function: > for example, 0 is a function 0/0. Is it right? > > Best Regards, > YHH. > > > > > > > -- > *Henning Diedrich* > CEO > > > > Eonblast Corporation > hdiedrich@REDACTED > +1.404.418.5002 w > www.eonblast.com > This email contains confidential and/or privileged information. If you are > not the intended recipient (or have received this email in error) please > notify the sender immediately and destroy this email. Any unauthorized > copying, disclosure or distribution of the material in this email is > prohibited. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Mon May 17 22:53:37 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 17 May 2010 22:53:37 +0200 Subject: [erlang-questions] Re: Defensive Programming In-Reply-To: References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> <4BF07BCF.7020309@eonblast.com> <4BF0E6D0.4010609@erlang-solutions.com> Message-ID: <4BF1ACD1.80002@eonblast.com> Thanks for the discussion Mazen! I like the motto of your blog, by the way. Exactly. Since I am writing functions for somebody else to use, it's different, even in two steps. First, I want to help the user if he makes wrong assumptions (yes, we had that before and yes I am maintaining I can pre-guess wrong assumptions) or mistakes. Then, I want to shield him from my own errors, too. (). I'll keep an eye on how this only makes things complicated. Thanks for your thoughts, Henning Robert Virding wrote: > I was going to say that you should handle the error where it is most > sensible to do so but you said it better. :-) > > Robert > > 2010/5/17 Mazen Harake : > >> I wrote this once... see if it makes sense: >> http://mazenharake.wordpress.com/2009/09/14/let-it-crash-the-right-way/ >> >> /Mazen >> >> >> On 17/05/2010 02:12, Henning Diedrich wrote: >> >>> Thanks Steve, >>> >>> that question of "is it recoverable?" makes a lot of sense to clarify. I >>> am taking note. >>> >>> In my case I am writing a lib function for others to use, which is exactly >>> why I would like to give them more information on the error. >>> >>> Even though they might not be able to recover it, they should at least >>> know why it didn't work, instead of just getting a bad match for "ok = " >>> >>> I meanwhile conclude that the trimmest variant will be to care less and >>> demand that the user of the function test for the return value directly >>> handed over from the gen_tcp:send() call. Making it >>> >>> login(Name, Password) -> >>> >>> ... >>> gen_tcp:send(Socket, term_to_binary(Str)). >>> >>> >>> >>> That looks the Erlangest an hands responsibility to the person who can >>> actually answer the 'recoverability question', the user of the lib. >>> >>> I realize I'd like to take away chances of frustration, puffing it up in >>> the process. >>> >>> Thanks for taking a look! >>> Henning >>> >>> >>> >>> Steve Davis wrote: >>> >>>> Hi Henning, >>>> >>>> This isn't intended to be any kind of definite answer, and I'm >>>> interested to see how others respond. >>>> >>>> In my case how I deal with this question is that I'd probably ask >>>> myself: >>>> >>>> *Is this exceptional case _recoverable_,? e.g. in the case of the tcp >>>> send, you could try to resend the data, so the first idiom may do it. >>>> >>>> If it's not recoverable, I'd just let it crash out. In fact I would >>>> likely force it to crash out using the second idiom. >>>> >>>> If I want to get fancy with error handling, I find that try..catch at >>>> the top level is usually enough. >>>> >>>> A further aside to this is that a certain Mr Armstrong once introduced >>>> me to an interesting "transactional" idiom also, namely: >>>> >>>> try begin >>>> ... >>>> end catch >>>> ... >>>> >>>> Regards, >>>> >>>> /s >>>> >>>> On May 16, 5:04 pm, Henning Diedrich wrote: >>>> >>>>> Hi list, >>>>> >>>>> I have a question about the Erlang way. >>>>> >>>>> I realize I keep programming defensive using exceptions/error calls to >>>>> *label failure* more precisely. >>>>> >>>>> E.g. ( - PRESUMABLY WRONG - ) >>>>> >>>>> login(Name, Password) -> >>>>> >>>>> ... >>>>> case >>>>> gen_tcp:send(Socket, term_to_binary(Str)) of >>>>> ok -> ok; >>>>> _ -> erlang:error(*sending_failed, {Str }*) >>>>> end >>>>> ... >>>>> >>>>> Instead of simply >>>>> >>>>> login(Name, Password) -> >>>>> >>>>> ... >>>>> ok = gen_tcp:send(Socket, term_to_binary(Str)), >>>>> ... >>>>> >>>>> It feels wrong in Erlang and I wonder if I am missing out on something. >>>>> >>>>> If somebody has a pointer to read up on this, thanks in advance, >>>>> >>>>> Henning >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Erlang Programming" group. >>>>> To post to this group, send email to >>>>> erlang-programming@REDACTED >>>>> To unsubscribe from this group, send email to >>>>> erlang-programming+unsubscribe@REDACTED >>>>> For more options, visit this group >>>>> athttp://groups.google.com/group/erlang-programming?hl=en. >>>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG >> SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > > From enewhuis@REDACTED Mon May 17 23:07:48 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Mon, 17 May 2010 16:07:48 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> Message-ID: <326A2A87-DB43-48AA-A96F-D0179819F3B5@gmail.com> I guess I like it. As easy as this is, I guess it is a sort of programming idiom. Although I loathe the introduction of temporary variables I think this suggestion is a good compromise. The one area where it still breaks down is when stringing two of these together. I'd have to use a different temporary variable name for each block of code. There is no LET construct in Erlang. :( On May 17, 2010, at 11:09 AM, Robert Raschke wrote: > What's wrong with these? So, your temp var is now called X instead of _ and > you always know what you're looking at. > > On Mon, May 17, 2010 at 3:00 PM, Eric Newhuis (personal) > wrote: > >> >> case some_module:some_function(...) of >> {some, pattern} -> _; >> {some, other, pattern} -> _; >> _ -> whatever >> end. >> >> > case X = some_module:some_function(...) of > {some, pattern} -> X; > {some, other, pattern} -> X; > _ -> whatever > end. > > > case some_module:some_function(...) of >> {some, pattern} -> >> {encapsulated, _}; >> {some, other, pattern} -> >> {another, _, encapsulation} >> end. >> >> > case X = some_module:some_function(...) of > {some, pattern} -> > {encapsulated, X}; > {some, other, pattern} -> > {another, X, encapsulation} > end. > > > case some_module:some_function(...) of >> {some, _, pattern} -> % _1 >> case _ of -> % _2 >> {some, great, pattern} -> >> not_so_bad; >> _ -> % _3 >> {_, Kind, _} = _, % _4, _5, _6 >> Kind >> end >> end. >> >> > case some_module:some_function(...) of > {some, X, pattern} -> % _1 > case X of -> % _2 > {some, great, pattern} -> > not_so_bad; > _ -> % _3 > {_, Kind, _} = X, % _4, _5, _6 > Kind > end > end. > > > Robby From chandrashekhar.mullaparthi@REDACTED Tue May 18 00:29:39 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 17 May 2010 23:29:39 +0100 Subject: ANNOUNCE: ibrowse-1.6.0 Message-ID: Hi all, ibrowse-1.6.0 is available on github. Extract from the README. 12-05-2010 - * Added support for the CONNECT method to tunnel HTTPS through a proxy. When a https URL is requested through a proxy, ibrowse will automatically use the CONNECT method to first setup a tunnel through the proxy. Once this succeeds, the actual request is dispatched. Successfully tested with the new SSL implementation in R13B-03 * Added SSL support for direct connections. See ibrowse:spawn_worker_process/1 and ibrowse:spawn_link_worker_process/1 * Added option to return raw status line and raw unparsed headers 23-04-2010 - * Fixes to URL parsing by Karol Skocik Git it from git://github.com/cmullaparthi/ibrowse.git cheers Chandru From ok@REDACTED Tue May 18 01:09:09 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 18 May 2010 11:09:09 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> Message-ID: <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> On May 18, 2010, at 2:00 AM, Eric Newhuis (personal) wrote: > For the record, I might still disagree, so far. I'm not sure. > Simply for argument's sake... > > The context in which this might be maximally useful is the following. > > case some_module:some_function(...) of > {some, pattern} -> _; > {some, other, pattern} -> _; > _ -> whatever > end. What is wrong with case X = some_module:some_function(...) of {some, pattern} -> X ; {some, other, pattern} -> X ; _ -> whatever end > > Note that I belong to the school of philosophy that suggests that > the number of temporary variables should be minimized. By using the wild card, you have *NOT* minimised the number of temporary variables. All you have done is to carefully deprive your reader of any clues as to what it is about. > I don't understand why the above would be called wild-card abuse. Because the whole *point* of the wild-card is that each and every occurrence of "_" should represent a DIFFERENT variable. That's what it means in Prolog, Mercury, Strand88, Parlog, GHC, ML, Haskell, Clean, ... Every time, a different variable. But you are relying on the "_" in each arm of the case being the *SAME* variable. You are also creating great confusion. Suppose I have case foo() of {ping,_} -> _ ; {_,pong} -> _ end The wild cards *following* the arrows are the *same* variable; what about the wild cards inside the patterns? If not, why not? What if I write case foo() of _ -> case bar() of _ -> _ end end Does this mean the same as case foo() of X -> case bar() of X -> X end end or case foo() of X -> case bar() of Y -> X end end or case foo() of X -> case bar() of Y -> Y end end or what? There's another point. In Erlang as it stands, _every_ variable _without exception_ must be visibly present at the point where it is bound. (Wild cards are no exceptions to this rule). You are introducing a new reading of "_" that violates this rule. If you want to think of the variable as bound in the pattern, write case Expr of X = Pattern when Guard -> X If you want to think of the variable as bound in the head, write case X = Expr of Pattern when Guard -> X In either case, the variable X is *visibly* bound. > It is clear from context that the wild-card represents something > other than matching. Yes, but we already *have* something we can use for this purpose. Ordinary variable names. > We've seen this idea before. There are those grammars that expose > variables whose value is whatever matched. I have no idea what you are referring to. Can you explain? "Exposing variables" is different from "invisibly binding anonymous variables". > > I suppose the following is where this great idea of mine might break > down. > > case some_module:some_function(...) of > {some, pattern} -> > {encapsulated, _}; > {some, other, pattern} -> > {another, _, encapsulation} > end. > > Although I still don't have a problem with that. You may not. I do. > From context I know that the right hand side of the arrow isn't > pattern matching. You as author may; your reader WILL have to work harder in reading to find out what the context *is*, especially if (as is often the case) the arrow is on the previous screen. This would be better as case X = some_module:some_function(...) of {some, pattern} -> {encapsulated, X} ; {some, other, pattern} -> {another, X, encapsulation} end There is not one of your examples that would not be massively improved by introducing a named variable, even a literal X. > > I guess where readability might break down is in nesting: > > case some_module:some_function(...) of > {some, _, pattern} -> % _1 > case _ of -> % _2 > {some, great, pattern} -> > not_so_bad; > _ -> % _3 > {_, Kind, _} = _, % _4, _5, _6 > Kind > end > end. > > Although I can still read the above once I learn that underscore > ('_') is context sensitive. > > _1 :: any() > _2 :: {some, any(), pattern} > _3 :: {some, any(), pattern}, not {some, great, pattern} > _4 :: some > _5 :: pattern > _6 :: _3 If I want to play at silly puzzles I do the Sudoku problem in the newspaper. If I'm reading a program, I do *NOT* enjoy pointless stumbling-blocks placed in my path to understanding. Let's rewrite your example. case some_module:some_function(...) of {some,Kind,pattern} -> case Kind of great -> not_so_bad _ -> Kind end end There is now *no* puzzle-solving for the reader. Let me raise my usual refrain: let's have a REAL example. From ok@REDACTED Tue May 18 01:35:42 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 18 May 2010 11:35:42 +1200 Subject: [erlang-questions] Does Erlang Treat Everything Function? In-Reply-To: References: Message-ID: <0F725B95-A42F-403B-849A-7DC685B553C3@cs.otago.ac.nz> On May 18, 2010, at 4:51 AM, ??? (Yau-Hsien Huang) wrote: > Hi, Erlang-lovers! I'm writing Erlang tutorials in my local > language, Chinese. I felt in thinking about a word > "Everything is functions." > Could I say that Erlang realizes the word too? > That is, to say even if a term is number, it's also a function: > for example, 0 is a function 0/0. Is it right? About a hundred years ago, people were trying to put mathematics on a rigorous foundation. After several attempts, consistent set theories were developed. In set theory, we *define* 0 = {} 1 = {0} 2 = {0,1} ... n = {0,1,2,...,n-1} with the result that i < j i ? j i ? j (that's proper-subset) all mean exactly the same thing for ordinal numbers. But outside set theory, nobody would think of 1 ? 2 as anything but a mistake. Church's lambda-calculus was part of the foundations programme. In the pure lambda calculus we can *define* 0 = \f.\x.x 1 = \f.\x.f x 2 = \f.\x.f (f x) ... n = \f.\x.f ( .... (f x) ...) f applied n times with the result that i ? j i ? j do the same. But outside pure lambda calculus, nobody would think of 3 ? 4 as anything but a mistake. There is a simple experiment you can perform to see whether in fact 0 is a function named 0 with arity 0 in Erlang. % erl 1> 0(). ** exception error: bad function 0 2> X = 0, X(). ** exception error: bad function 0 Erlang also draws a distinction between atoms and functions of no arguments: 3> q. q 4> q(). ok 5> % (Unix prompt; q() shut Erlang down) For practical reasons, functional programming languages like Lisp, Erlang, Haskell, Clean, and ML DON'T take the view that everything is a function. Most grammatically possible programs are nonsense. The statically typed languages try to reject as much nonsense as possible at compile time, at the price of rejecting some things that are not nonsense. In those languages, numbers *might* well be functions really, but they ensure that you can't *tell* whether they are or not. The dynamically typed languages have to wait until run time to check for probable nonsense. Things that it is useful to distinguish (like numbers and atoms) are, as it were, painted in different colours. The number 0 and the atom '0' might well *be* functions underneath, and they might well even be the same function, but 0+'0' finds one thing with number- coloured paint and another with atom-coloured paint and exclaims in horror. From silent_vendetta@REDACTED Tue May 18 01:32:16 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 17 May 2010 16:32:16 -0700 Subject: [erlang-questions] Mnesia questions In-Reply-To: <4BF16B65.9090607@erlang-solutions.com> References: ,<4BF16B65.9090607@erlang-solutions.com> Message-ID: First off thank you to everyone who responded publicly and privately. I do have a couple more specific ones about Mnesia and fragmented tables (yes, more of THOSE questions). 1) Well my first question is actually about tables in general. When one inserts/deletes a record/row from a table, is the whole table locked(didn't see this anywhere but could have missed it)? 2) I understand the general concept behind fragmented tables, and the answer to the above question may render part of this moot, but what are the performance pros/cons for using fragmented tables? I don't mean disk space but read/writes which occur mostly in transactions. Is each one any slower? Is there any advantage in parallelization? 2a)If a whole table is locked for an update does that just mean (hopefully) that the specific fragment of the larger table would only be locked, meaning a 50 fragment table could have 50 different locks/writes going? Of course if a whole table is not locked for an update that doesn't matter. One of the data requirements for my project is that one "person" record will be associated with possibly as many as 1000 "item" records. Lets take a possible person population of 10,000, meaning there could be as many as 10M item records. I want to keep each record small as I will be accessing different parts of a whole "object" at different times and will only need one or two small pieces of data at a time. One of the ways, and some of this is going to rely on the answers above, I was thinking of doing this was setting up a bag type table and associating all of those items (because they will be different) with the same key, the id of the person record. The one thing I'm wondering about, however, is some of these items might be in table fragment 1 and others in fragments 13, 27 and 42 (as far as I understand it) and I was wondering how much of a problem this would create as far as performance. Having each item with a different primary key, with each key held in a field in the person record, would mean I have a ton of queries if I want to get detailed info about each item and that doesn't work for me. Is there another approach that would be better that I am not thinking of? _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 From hd2010@REDACTED Tue May 18 02:01:33 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 18 May 2010 02:01:33 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> Message-ID: <4BF1D8DD.8080000@eonblast.com> But if it was another symbol? Say ~. Richard O'Keefe wrote: > > On May 18, 2010, at 2:00 AM, Eric Newhuis (personal) wrote: > >> For the record, I might still disagree, so far. I'm not sure. >> Simply for argument's sake... >> >> The context in which this might be maximally useful is the following. >> >> case some_module:some_function(...) of >> {some, pattern} -> _; >> {some, other, pattern} -> _; >> _ -> whatever >> end. > > What is wrong with > > case X = some_module:some_function(...) > of {some, pattern} -> X > ; {some, other, pattern} -> X > ; _ -> whatever > end >> >> Note that I belong to the school of philosophy that suggests that the >> number of temporary variables should be minimized. > > By using the wild card, you have *NOT* minimised the number of temporary > variables. All you have done is to carefully deprive your reader of any > clues as to what it is about. > > >> I don't understand why the above would be called wild-card abuse. > > Because the whole *point* of the wild-card is that each and > every occurrence of "_" should represent a DIFFERENT variable. > That's what it means in Prolog, Mercury, Strand88, Parlog, GHC, > ML, Haskell, Clean, ... Every time, a different variable. > But you are relying on the "_" in each arm of the case being > the *SAME* variable. > > You are also creating great confusion. > Suppose I have > > case foo() > of {ping,_} -> _ > ; {_,pong} -> _ > end > > The wild cards *following* the arrows are the *same* variable; > what about the wild cards inside the patterns? If not, why not? > > What if I write > case foo() of _ -> case bar() of _ -> _ end end > Does this mean the same as > case foo() of X -> case bar() of X -> X end end > or case foo() of X -> case bar() of Y -> X end end > or case foo() of X -> case bar() of Y -> Y end end > or what? > > There's another point. In Erlang as it stands, _every_ > variable _without exception_ must be visibly present at > the point where it is bound. (Wild cards are no exceptions > to this rule). You are introducing a new reading of "_" > that violates this rule. If you want to think of the > variable as bound in the pattern, write > case Expr > of X = Pattern when Guard -> X > If you want to think of the variable as bound in the head, > write > case X = Expr > of Pattern when Guard -> X > In either case, the variable X is *visibly* bound. > >> It is clear from context that the wild-card represents something >> other than matching. > > Yes, but we already *have* something we can use for this purpose. > Ordinary variable names. > > >> We've seen this idea before. There are those grammars that expose >> variables whose value is whatever matched. > > I have no idea what you are referring to. Can you explain? > > "Exposing variables" is different from "invisibly binding anonymous > variables". > >> >> I suppose the following is where this great idea of mine might break >> down. >> >> case some_module:some_function(...) of >> {some, pattern} -> >> {encapsulated, _}; >> {some, other, pattern} -> >> {another, _, encapsulation} >> end. >> >> Although I still don't have a problem with that. > > You may not. I do. > >> From context I know that the right hand side of the arrow isn't >> pattern matching. > > You as author may; your reader WILL have to work harder in reading > to find out what the context *is*, especially if (as is often the > case) the arrow is on the previous screen. > > This would be better as > > case X = some_module:some_function(...) > of {some, pattern} -> > {encapsulated, X} > ; {some, other, pattern} -> > {another, X, encapsulation} > end > > There is not one of your examples that would not be massively improved > by introducing a named variable, even a literal X. >> >> I guess where readability might break down is in nesting: >> >> case some_module:some_function(...) of >> {some, _, pattern} -> % _1 >> case _ of -> % _2 >> {some, great, pattern} -> >> not_so_bad; >> _ -> % _3 >> {_, Kind, _} = _, % _4, _5, _6 >> Kind >> end >> end. >> >> Although I can still read the above once I learn that underscore >> ('_') is context sensitive. >> >> _1 :: any() >> _2 :: {some, any(), pattern} >> _3 :: {some, any(), pattern}, not {some, great, pattern} >> _4 :: some >> _5 :: pattern >> _6 :: _3 > > If I want to play at silly puzzles I do the Sudoku problem in the > newspaper. If I'm reading a program, I do *NOT* enjoy pointless > stumbling-blocks placed in my path to understanding. > > Let's rewrite your example. > > case some_module:some_function(...) > of {some,Kind,pattern} -> > case Kind > of great -> not_so_bad > _ -> Kind > end > end > > There is now *no* puzzle-solving for the reader. > > Let me raise my usual refrain: > let's have a REAL example. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From igorrs@REDACTED Tue May 18 02:35:35 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 17 May 2010 21:35:35 -0300 Subject: [erlang-questions] Mnesia questions In-Reply-To: References: <4BF16B65.9090607@erlang-solutions.com> Message-ID: To model your data, I think it's a good idea to first decide what are the operations you're going to need. Also, I can't understand whether each item is specific of a person or the same item can belong to more than one person. Supposing that each item belongs to only one person and that you need to be able to: - Retrieve all the items of a person. - Retrieve a specific item of a person. - Insert an item for a person. I think you would be fine with a table that has: - {PersonKey, ItemKey} as the primary key. - Another field for PersonKey. - An index on the field above. This way, the items of the same user may be distributed among several fragments, what is probably what you want. Using a bag with the person as the primary key, all the items for the same user will be on the same fragment. Also, accessing a specific item will be slower. Good luck. Igor. On Mon, May 17, 2010 at 8:32 PM, Chris Hicks wrote: > > First off thank you to everyone who responded publicly and privately. I do have a couple more specific ones about Mnesia and fragmented tables (yes, more of THOSE questions). > 1) Well my first question is actually about tables in general. When one inserts/deletes a record/row from a table, is the whole table locked(didn't see this anywhere but could have missed it)? > 2) I understand the general concept behind fragmented tables, and the answer to the above question may render part of this moot, but what are the performance pros/cons for using fragmented tables? I don't mean disk space but read/writes which occur mostly in transactions. Is each one any slower? Is there any advantage in parallelization? > 2a)If a whole table is locked for an update does that just mean (hopefully) that the specific fragment of the larger table would only be locked, meaning a 50 fragment table could have 50 different locks/writes going? Of course if a whole table is not locked for an update that doesn't matter. > One of the data requirements for my project is that one "person" record will be associated with possibly as many as 1000 "item" records. Lets take a possible person population of 10,000, meaning there could be as many as 10M item records. I want to keep each record small as I will be accessing different parts of a whole "object" at different times and will only need one or two small pieces of data at a time. One of the ways, and some of this is going to rely on the answers above, I was thinking of doing this was setting up a bag type table and associating all of those items (because they will be different) with the same key, the id of the person record. The one thing I'm wondering about, however, is some of these items might be in table fragment 1 and others in fragments 13, 27 and 42 (as far as I understand it) and I was wondering how much of a problem this would create as far as performance. > Having each item with a different primary key, with each key held in a field in the person record, would mean I have a ton of queries if I want to get detailed info about each item and that doesn't work for me. Is there another approach that would be better that I am not thinking of? From timuckun@REDACTED Tue May 18 02:35:55 2010 From: timuckun@REDACTED (Tim Uckun) Date: Tue, 18 May 2010 12:35:55 +1200 Subject: Library management in erlang. Message-ID: Is there anything equivalent to ruby gems in erlang? I am trying to follow a couple of tutorials they all say "go download this library, git clone this library" etc. How do you guys keep your libraries current and secure? is there an equivalent of CPAN or rubyforge where I can search for libraries that might be of use? Are there the equivalent of gem search, gem list, gem install, gem update,gem uninstall etc. From steven.charles.davis@REDACTED Tue May 18 03:18:16 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 17 May 2010 18:18:16 -0700 (PDT) Subject: Defensive Programming In-Reply-To: <4BF1ACD1.80002@eonblast.com> References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> <4BF07BCF.7020309@eonblast.com> <4BF0E6D0.4010609@erlang-solutions.com> <4BF1ACD1.80002@eonblast.com> Message-ID: Another thought that may be useful... I've found that a _really_ good way of working out how to organize a library api is to code two or three (reasonably real) apps against it. When I do it that way, I find the worst holes in the api pretty quickly, and often find that my initial guesses were not the most important concerns at all. regs, /s (btw very interesting blog post mazen, i hadn't seen that before) On May 17, 3:53?pm, Henning Diedrich wrote: > Thanks for the discussion Mazen! I like the motto of your blog, by the > way. Exactly. > > Since I am writing functions for somebody else to use, it's different, > even in two steps. > > First, I want to help the user if he makes wrong assumptions (yes, we > had that before and yes I am maintaining I can pre-guess wrong > assumptions) or mistakes. Then, I want to shield him from my own errors, > too. (). > > I'll keep an eye on how this only makes things complicated. > > Thanks for your thoughts, > Henning > > > > > > Robert Virding wrote: > > I was going to say that you should handle the error where it is most > > sensible to do so but you said it better. :-) > > > Robert > > > 2010/5/17 Mazen Harake : > > >> I wrote this once... see if it makes sense: > >>http://mazenharake.wordpress.com/2009/09/14/let-it-crash-the-right-way/ > > >> /Mazen > > >> On 17/05/2010 02:12, Henning Diedrich wrote: > > >>> Thanks Steve, > > >>> that question of "is it recoverable?" makes a lot of sense to clarify. I > >>> am taking note. > > >>> In my case I am writing a lib function for others to use, which is exactly > >>> why I would like to give them more information on the error. > > >>> Even though they might not be able to recover it, they should at least > >>> know why it didn't work, instead of just getting a bad match for "ok = " > > >>> I meanwhile conclude that the trimmest variant will be to care less and > >>> demand that the user of the function test for the return value directly > >>> handed over from the gen_tcp:send() call. Making it > > >>> login(Name, Password) -> > > >>> ? ? ?... > >>> ? ? ? gen_tcp:send(Socket, term_to_binary(Str)). > > >>> That looks the Erlangest an hands responsibility to the person who can > >>> actually answer the 'recoverability question', the user of the lib. > > >>> I realize I'd like to take away chances of frustration, puffing it up in > >>> the process. > > >>> Thanks for taking a look! > >>> Henning > > >>> Steve Davis wrote: > > >>>> Hi Henning, > > >>>> This isn't intended to be any kind of definite answer, and I'm > >>>> interested to see how others respond. > > >>>> In my case how I deal with this question is that I'd probably ask > >>>> myself: > > >>>> *Is this exceptional case _recoverable_,? e.g. in the case of the tcp > >>>> send, you could try to resend the data, so the first idiom may do it. > > >>>> If it's not recoverable, I'd just let it crash out. In fact I would > >>>> likely force it to crash out using the second idiom. > > >>>> If I want to get fancy with error handling, I find that try..catch at > >>>> the top level is usually enough. > > >>>> A further aside to this is that a certain Mr Armstrong once introduced > >>>> me to an interesting "transactional" idiom also, namely: > > >>>> try begin > >>>> ... > >>>> end catch > >>>> ... > > >>>> Regards, > > >>>> /s > > >>>> On May 16, 5:04 pm, Henning Diedrich wrote: > > >>>>> Hi list, > > >>>>> I have a question about the Erlang way. > > >>>>> I realize I keep programming defensive using exceptions/error calls to > >>>>> *label failure* more precisely. > > >>>>> E.g. ( - PRESUMABLY WRONG - ) > > >>>>> login(Name, Password) -> > > >>>>> ? ? ? ... > >>>>> ? ? ? ?case > >>>>> ? ? ? ? ? ? ? ?gen_tcp:send(Socket, term_to_binary(Str)) of > >>>>> ? ? ? ? ? ? ? ? ? ? ok -> ok; > >>>>> ? ? ? ? ? ? ? ? ? ? _ -> erlang:error(*sending_failed, {Str }*) > >>>>> ? ? ? end > >>>>> ? ? ? ... > > >>>>> Instead of simply > > >>>>> login(Name, Password) -> > > >>>>> ? ? ? ... > >>>>> ? ? ? ?ok = gen_tcp:send(Socket, term_to_binary(Str)), > >>>>> ? ? ? ... > > >>>>> It feels wrong in Erlang and I wonder if I am missing out on something. > > >>>>> If somebody has a pointer to read up on this, thanks in advance, > > >>>>> Henning > > >>>>> -- > >>>>> You received this message because you are subscribed to the Google > >>>>> Groups "Erlang Programming" group. > >>>>> To post to this group, send email to > >>>>> erlang-programming@REDACTED > >>>>> To unsubscribe from this group, send email to > >>>>> erlang-programming+unsubscribe@REDACTED > >>>>> For more options, visit this group > >>>>> athttp://groups.google.com/group/erlang-programming?hl=en. > > >>>> ________________________________________________________________ > >>>> erlang-questions (at) erlang.org mailing list. > >>>> Seehttp://www.erlang.org/faq.html > >>>> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > >> --------------------------------------------------- > > >> --------------------------------------------------- > > >> WE'VE CHANGED NAMES! > > >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG > >> SOLUTIONS LTD. > > >>www.erlang-solutions.com > > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> Seehttp://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From steven.charles.davis@REDACTED Tue May 18 03:22:42 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 17 May 2010 18:22:42 -0700 (PDT) Subject: Library management in erlang. In-Reply-To: References: Message-ID: Closest I have seen are: http://projects.trapexit.org/web/ http://jungerl.sourceforge.net/ http://erlware.org/ /s On May 17, 7:35?pm, Tim Uckun wrote: > Is there anything equivalent to ruby gems in erlang? > > I am trying to follow a couple of tutorials they all say "go download > this library, git clone this library" etc. ?How do you guys keep your > libraries current and secure? is there an equivalent of CPAN or > rubyforge where I can search for libraries that might be of use? > > Are there the equivalent of ?gem search, gem list, gem install, gem > update,gem uninstall etc. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. From ok@REDACTED Tue May 18 03:43:41 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 18 May 2010 13:43:41 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4BF1D8DD.8080000@eonblast.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <4BF1D8DD.8080000@eonblast.com> Message-ID: <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> On May 18, 2010, at 12:01 PM, Henning Diedrich wrote: > But if it was another symbol? Say ~. No, let's NOT say \sim, a symbol widely used in mathematics and physics for something that is syntactically a binary relation and means "sort-of equal, kinda", which is why I want it for frames, and is used in a variety of programming languages as a unary operator with negation-related meanings. But what if it were some other symbol? Well, WHY a symbol? What's the difference between @#$%^ and Fred? The differences are clear: + there is an infinite set of variable names, which can be chosen to be easy to pronounce and understand, allowing you to name many things at the same time - a symbol is one thing, probably doesn't have an agreed way to say it (is ~ "tilde", "wiggle", "squiggle", "sim", "roughly equals", or what) and has no special connexion with what it stands for; since there's one of it, it's very hard to use in a nested context. Example: Common Lisp has * the value of the last expression you typed in ** the old value of * *** the old value of ** What happens if you want to refer to the old value of *** ? Tough luck. What happens if you miscount and typed ** when you meant *** ? Tough luck. What is the connection between multiplication and history that makes this memorable? There isn't one. (OK: + ++ +++ are the last three forms you typed, * ** *** are their (primary) values / // /// are the lists of all their values - is the form that is being evaluated right now How "-" gets to be more recent than "+" is a mystery, but at least they managed to blow this up into an "arithmetic function = history variable" analogy. The consistency is perfect. < << and <<< do the obvious things. No, wait, there _is_ no obvious thing, and if there were, they don't. Oh well, at least there are consistently X, XX, and XXX versions. No, wait, there aren't. -- and --- do not exist.) Why invent triangular wheels covered in barbed wire and razor blades when we have nice round rubber ones? From silent_vendetta@REDACTED Tue May 18 04:02:26 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 17 May 2010 19:02:26 -0700 Subject: [erlang-questions] Mnesia questions In-Reply-To: References: <4BF16B65.9090607@erlang-solutions.com>,, Message-ID: I'm not sure I understand exactly what you are describing for the table you recommend, I think the dual PersonKey is what is throwing me off. What you are recommending is a separate item table, fragmented, where the primary key is the name of that separate (from the person table) table and the specific item ID. Then putting in another field that contains the ID of the associated person record in the person table and indexing on that. This way each item can be looked up singularly or as the group that belongs to that single player. Correct? What is the advantage to having the items spread out over multiple fragments as opposed to having them all located in a single table in the case of a bag? > From: igorrs@REDACTED > Date: Mon, 17 May 2010 21:35:35 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Mnesia questions > > To model your data, I think it's a good idea to first decide what are > the operations you're going to need. > > Also, I can't understand whether each item is specific of a person or > the same item can belong to more than one person. > > Supposing that each item belongs to only one person and that you need > to be able to: > - Retrieve all the items of a person. > - Retrieve a specific item of a person. > - Insert an item for a person. > > I think you would be fine with a table that has: > - {PersonKey, ItemKey} as the primary key. > - Another field for PersonKey. > - An index on the field above. > > This way, the items of the same user may be distributed among several > fragments, what is probably what you want. > > Using a bag with the person as the primary key, all the items for the > same user will be on the same fragment. Also, accessing a specific > item will be slower. > > Good luck. > Igor. > > On Mon, May 17, 2010 at 8:32 PM, Chris Hicks > wrote: > > > > First off thank you to everyone who responded publicly and privately. I do have a couple more specific ones about Mnesia and fragmented tables (yes, more of THOSE questions). > > 1) Well my first question is actually about tables in general. When one inserts/deletes a record/row from a table, is the whole table locked(didn't see this anywhere but could have missed it)? > > 2) I understand the general concept behind fragmented tables, and the answer to the above question may render part of this moot, but what are the performance pros/cons for using fragmented tables? I don't mean disk space but read/writes which occur mostly in transactions. Is each one any slower? Is there any advantage in parallelization? > > 2a)If a whole table is locked for an update does that just mean (hopefully) that the specific fragment of the larger table would only be locked, meaning a 50 fragment table could have 50 different locks/writes going? Of course if a whole table is not locked for an update that doesn't matter. > > One of the data requirements for my project is that one "person" record will be associated with possibly as many as 1000 "item" records. Lets take a possible person population of 10,000, meaning there could be as many as 10M item records. I want to keep each record small as I will be accessing different parts of a whole "object" at different times and will only need one or two small pieces of data at a time. One of the ways, and some of this is going to rely on the answers above, I was thinking of doing this was setting up a bag type table and associating all of those items (because they will be different) with the same key, the id of the person record. The one thing I'm wondering about, however, is some of these items might be in table fragment 1 and others in fragments 13, 27 and 42 (as far as I understand it) and I was wondering how much of a problem this would create as far as performance. > > Having each item with a different primary key, with each key held in a field in the person record, would mean I have a ton of queries if I want to get detailed info about each item and that doesn't work for me. Is there another approach that would be better that I am not thinking of? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 From hd2010@REDACTED Tue May 18 05:05:05 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 18 May 2010 05:05:05 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <4BF1D8DD.8080000@eonblast.com> <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> Message-ID: <4BF203E1.9070905@eonblast.com> Richard, righteous, I was just curious about the defense you'd mount to keep ~ for the frame syntax. Where the first thing that hit my uneducated I was how the use of ~ in it was so completely special --- as some other symbol uses in Erlang are --- that it, well, may be an excellent fit. Why starting to get conventional. A language proudly flying =< could also have ~ to mean 'something like' while really, yes, significantly different from =. I think I already got used to the proposed frame syntax, followed your arguments and support it. But I think ~ would be great to also roughly mean 'what I just said'. Also ./. might do, or -"-, or *. The argument about being hard to pronounce sounds a bit convenient in this context. Pronouncing code is a rarified art and runs into this problem time again. There'll be a suggested way to say it and then that is that. Some suggestions below. Or was the pronunciation of frame definitions already decided upon? I think 'sorta-kinda' for ~, as you proposed, may be just fine. It has this ring of order to it (sort-a), and of being nice (kind), which will help it be easily remembered. Things like this count! Plus, it will also work for frame definitions where you pointed out that 'equals' would be awfully wrong. But just as '=' can mean many things to different people, maybe '~' can be used for both things. I thought it was a tad illogical, however, to argue that because there is no obviousness to the meaning of <<< in Lisp might indicate that the 'some as above/before' connotation of ~ should likewise and somehow guilty by association be irrelevant or even counterproductive when deciding about. It's use. Not that I believe that anyone cares, but eye have a hobby horse that had me thinking about hacking the pre-compiler already: the doubling of function signatures where the variation that is /called/ in the former is, plus one default argument as literal, exactly the head of the next. I might be doing something wrong there but I keep copying heads and it makes the code hard to read. I regularily go, copy a head, paste it three times (one over the old ) and alter two of them marginally. A nice remedy could be: Fun(A) -> Fun(A, b). Fun(A,B) ... This could become Fun(A) -> ~~ b. Fun(A,B) ... Where ~~ would mean, 'the thing I just wrote, in and out of the brackets, and add this parameter as last argument'. It could be pronounced 'same-but-with' or 'razor', because that's really what it does, razor some boilerplate and double expressiveness, no? It could not be used every time, but could clean up some code. ~~~ (3x) finally, could be used to be expanded internally into _ -> ~, or, X -> X and could be called 'tri-barbed-wire' as I think you suggested. I can pronounce that. Sorry there is no relation to the negation connotation that ~ somehow brings along. But that seemed to be alright for its use in io:format in the first place, didn't it? Also, somehow, it negates writing code doubly. Yours sincerely, Henning Richard O'Keefe wrote: > > On May 18, 2010, at 12:01 PM, Henning Diedrich wrote: > >> But if it was another symbol? Say ~. > > No, let's NOT say \sim, a symbol widely used in mathematics and > physics for something that is syntactically a binary relation > and means "sort-of equal, kinda", which is why I want it for > frames, and is used in a variety of programming languages as a > unary operator with negation-related meanings. > > But what if it were some other symbol? > Well, WHY a symbol? What's the difference between @#$%^ and Fred? > The differences are clear: > > + there is an infinite set of variable names, which can be > chosen to be easy to pronounce and understand, allowing you > to name many things at the same time > > - a symbol is one thing, probably doesn't have an agreed way > to say it (is ~ "tilde", "wiggle", "squiggle", "sim", > "roughly equals", or what) and has no special connexion with > what it stands for; since there's one of it, it's very hard to > use in a nested context. > > Example: Common Lisp has > * the value of the last expression you typed in > ** the old value of * > *** the old value of ** > What happens if you want to refer to the old value of *** ? > Tough luck. What happens if you miscount and typed ** when > you meant *** ? Tough luck. What is the connection between > multiplication and history that makes this memorable? > There isn't one. > > (OK: + ++ +++ are the last three forms you typed, > * ** *** are their (primary) values > / // /// are the lists of all their values > - is the form that is being evaluated right now > How "-" gets to be more recent than "+" is a mystery, > but at least they managed to blow this up into an > "arithmetic function = history variable" analogy. > The consistency is perfect. < << and <<< do the > obvious things. No, wait, there _is_ no obvious > thing, and if there were, they don't. Oh well, > at least there are consistently X, XX, and XXX versions. > No, wait, there aren't. -- and --- do not exist.) > > > Why invent triangular wheels covered in barbed wire and razor blades > when we have nice round rubber ones? > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From enewhuis@REDACTED Tue May 18 05:25:18 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Mon, 17 May 2010 22:25:18 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> Message-ID: The specific case I've had a number of times is when writing short filter functions that either pass through or modify something. I'd find myself almost forced to write the following: case find(...) of error -> {ok, Default}; {ok, Value} -> {ok, Value} end. And the reconstruction of a small tuple bugged me. Your suggestion shortens this to something arguably easier to type. case X = find(...) of error -> {ok, Default}; {ok, Value} -> X end. ...without the apparent overhead of reconstruction. (Is it a moot point due to a clever compiler?) But I guess I am still concerned about suggesting this as a general programming idiom because it introduces a temporary variable. And as such one would need to study pre- and post-logic to avoid accidental use of the same variable name to name something different. I'd not have this concern if Erlang had a LET construct: let X = find(...) in case X of error -> {ok, Default}; {ok, Value} -> X end. Variable scope is important. Of course...absurdly... (fun() -> case X=find(...) of error -> {ok, Default1}; {ok, Value} -> X end end)(), (fun() -> case X=obviously_different_fetch(...) of error -> {ok, Default2}; {ok, Value} -> X end end)(). But that is ridiculous. So apparently my idea isn't so great. And I also don't have a satisfying temporary-free and reconstruction-free solution. On May 17, 2010, at 6:09 PM, Richard O'Keefe wrote: > > On May 18, 2010, at 2:00 AM, Eric Newhuis (personal) wrote: > >> For the record, I might still disagree, so far. I'm not sure. Simply for argument's sake... >> >> The context in which this might be maximally useful is the following. >> >> case some_module:some_function(...) of >> {some, pattern} -> _; >> {some, other, pattern} -> _; >> _ -> whatever >> end. > > What is wrong with > > case X = some_module:some_function(...) > of {some, pattern} -> X > ; {some, other, pattern} -> X > ; _ -> whatever > end >> >> Note that I belong to the school of philosophy that suggests that the number of temporary variables should be minimized. > > By using the wild card, you have *NOT* minimised the number of temporary > variables. All you have done is to carefully deprive your reader of any > clues as to what it is about. > > >> I don't understand why the above would be called wild-card abuse. > > Because the whole *point* of the wild-card is that each and > every occurrence of "_" should represent a DIFFERENT variable. > That's what it means in Prolog, Mercury, Strand88, Parlog, GHC, > ML, Haskell, Clean, ... Every time, a different variable. > But you are relying on the "_" in each arm of the case being > the *SAME* variable. > > You are also creating great confusion. > Suppose I have > > case foo() > of {ping,_} -> _ > ; {_,pong} -> _ > end > > The wild cards *following* the arrows are the *same* variable; > what about the wild cards inside the patterns? If not, why not? > > What if I write > case foo() of _ -> case bar() of _ -> _ end end > Does this mean the same as > case foo() of X -> case bar() of X -> X end end > or case foo() of X -> case bar() of Y -> X end end > or case foo() of X -> case bar() of Y -> Y end end > or what? > > There's another point. In Erlang as it stands, _every_ > variable _without exception_ must be visibly present at > the point where it is bound. (Wild cards are no exceptions > to this rule). You are introducing a new reading of "_" > that violates this rule. If you want to think of the > variable as bound in the pattern, write > case Expr > of X = Pattern when Guard -> X > If you want to think of the variable as bound in the head, > write > case X = Expr > of Pattern when Guard -> X > In either case, the variable X is *visibly* bound. > >> It is clear from context that the wild-card represents something other than matching. > > Yes, but we already *have* something we can use for this purpose. > Ordinary variable names. > > >> We've seen this idea before. There are those grammars that expose variables whose value is whatever matched. > > I have no idea what you are referring to. Can you explain? > > "Exposing variables" is different from "invisibly binding anonymous > variables". > >> >> I suppose the following is where this great idea of mine might break down. >> >> case some_module:some_function(...) of >> {some, pattern} -> >> {encapsulated, _}; >> {some, other, pattern} -> >> {another, _, encapsulation} >> end. >> >> Although I still don't have a problem with that. > > You may not. I do. > >> From context I know that the right hand side of the arrow isn't pattern matching. > > You as author may; your reader WILL have to work harder in reading > to find out what the context *is*, especially if (as is often the > case) the arrow is on the previous screen. > > This would be better as > > case X = some_module:some_function(...) > of {some, pattern} -> > {encapsulated, X} > ; {some, other, pattern} -> > {another, X, encapsulation} > end > > There is not one of your examples that would not be massively improved > by introducing a named variable, even a literal X. >> >> I guess where readability might break down is in nesting: >> >> case some_module:some_function(...) of >> {some, _, pattern} -> % _1 >> case _ of -> % _2 >> {some, great, pattern} -> >> not_so_bad; >> _ -> % _3 >> {_, Kind, _} = _, % _4, _5, _6 >> Kind >> end >> end. >> >> Although I can still read the above once I learn that underscore ('_') is context sensitive. >> >> _1 :: any() >> _2 :: {some, any(), pattern} >> _3 :: {some, any(), pattern}, not {some, great, pattern} >> _4 :: some >> _5 :: pattern >> _6 :: _3 > > If I want to play at silly puzzles I do the Sudoku problem in the > newspaper. If I'm reading a program, I do *NOT* enjoy pointless > stumbling-blocks placed in my path to understanding. > > Let's rewrite your example. > > case some_module:some_function(...) > of {some,Kind,pattern} -> > case Kind > of great -> not_so_bad > _ -> Kind > end > end > > There is now *no* puzzle-solving for the reader. > > Let me raise my usual refrain: > let's have a REAL example. > > From igorrs@REDACTED Tue May 18 08:08:52 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Tue, 18 May 2010 03:08:52 -0300 Subject: [erlang-questions] Mnesia questions In-Reply-To: References: <4BF16B65.9090607@erlang-solutions.com> Message-ID: On Mon, May 17, 2010 at 11:02 PM, Chris Hicks wrote: > I'm not sure I understand exactly what you are describing for the table you > recommend, I think the dual PersonKey is what is throwing me off. What you > are recommending is a separate item table, fragmented, where the primary key > is the name of that separate (from the person table) table and the specific > item ID. Then putting in another field that contains the ID of the > associated person record in the person table and indexing on that. This way > each item can be looked up singularly or as the group that belongs to that > single player. Correct? > What is the advantage to having the items spread out over multiple fragments > as opposed to having them all located in a single table in the case of a > bag? I was making some assumptions here (which may be wrong): - That you are using fragmented tables anyway. - That you are spreading your fragments across several nodes. - That, if one of the nodes is temporarily unavailable, you prefer to serve partial data to many "person"s than to serve nothing to some and everything to some other. As for the advantages of using fragmented tables, I can only think of two now: - Being able to spread your data across several servers. One can also say that having data spread across different files will also improve the fault tolerance of your system. - Overcoming the 2 GB limit of dets (or not allowing dets files to grow too big, for performance reasons). I should also point that I have both variants in production (using bags and using the scheme I described above), both holding big amounts of data (I can't say numbers, but it's more than what you described) and both using dets (since the data doesn't fit in memory, even with several servers). If I could go back in time, I wouldn't be using bags. Although they give me a little bit more performance (but again: all of this depends on the operations you need to do) and save some space, I strongly prefer to not have users with all of their data unavailable because one server is down (it hasn't happened so far, but I already hate bags ;-)). Best regards. Igor. >> From: igorrs@REDACTED >> Date: Mon, 17 May 2010 21:35:35 -0300 >> To: silent_vendetta@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Mnesia questions >> >> To model your data, I think it's a good idea to first decide what are >> the operations you're going to need. >> >> Also, I can't understand whether each item is specific of a person or >> the same item can belong to more than one person. >> >> Supposing that each item belongs to only one person and that you need >> to be able to: >> - Retrieve all the items of a person. >> - Retrieve a specific item of a person. >> - Insert an item for a person. >> >> I think you would be fine with a table that has: >> - {PersonKey, ItemKey} as the primary key. >> - Another field for PersonKey. >> - An index on the field above. >> >> This way, the items of the same user may be distributed among several >> fragments, what is probably what you want. >> >> Using a bag with the person as the primary key, all the items for the >> same user will be on the same fragment. Also, accessing a specific >> item will be slower. >> >> Good luck. >> Igor. >> >> On Mon, May 17, 2010 at 8:32 PM, Chris Hicks >> wrote: >> > >> > First off thank you to everyone who responded publicly and privately. I >> > do have a couple more specific ones about Mnesia and fragmented tables (yes, >> > more of THOSE questions). >> > 1) Well my first question is actually about tables in general. When one >> > inserts/deletes a record/row from a table, is the whole table locked(didn't >> > see this anywhere but could have missed it)? >> > 2) I understand the general concept behind fragmented tables, and the >> > answer to the above question may render part of this moot, but what are the >> > performance pros/cons for using fragmented tables? I don't mean disk space >> > but read/writes which occur mostly in transactions. Is each one any slower? >> > Is there any advantage in parallelization? >> > 2a)If a whole table is locked for an update does that just mean >> > (hopefully) that the specific fragment of the larger table would only be >> > locked, meaning a 50 fragment table could have 50 different locks/writes >> > going? Of course if a whole table is not locked for an update that doesn't >> > matter. >> > One of the data requirements for my project is that one "person" record >> > will be associated with possibly as many as 1000 "item" records. Lets take a >> > possible person population of 10,000, meaning there could be as many as 10M >> > item records. I want to keep each record small as I will be accessing >> > different parts of a whole "object" at different times and will only need >> > one or two small pieces of data at a time. One of the ways, and some of this >> > is going to rely on the answers above, I was thinking of doing this was >> > setting up a bag type table and associating all of those items (because they >> > will be different) with the same key, the id of the person record. The one >> > thing I'm wondering about, however, is some of these items might be in table >> > fragment 1 and others in fragments 13, 27 and 42 (as far as I understand it) >> > and I was wondering how much of a problem this would create as far as >> > performance. >> > Having each item with a different primary key, with each key held in a >> > field in the person record, would mean I have a ton of queries if I want to >> > get detailed info about each item and that doesn't work for me. Is there >> > another approach that would be better that I am not thinking of? >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________ > The New Busy is not the too busy. Combine all your e-mail accounts with > Hotmail. Get busy. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From silent_vendetta@REDACTED Tue May 18 09:36:53 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Tue, 18 May 2010 00:36:53 -0700 Subject: [erlang-questions] Mnesia questions In-Reply-To: References: <4BF16B65.9090607@erlang-solutions.com>, ,, Message-ID: Well for this project, while I doubt there will be more than one server, it is a sort of test-bed for launching a much more ambitious project sometime in the future that would span a number of servers. So while not all of what you have pointed out affects this specific project they are good things to keep in mind as I move forward and I can still structure my data in these ways to learn more about what it would be like in a much more demanding atmosphere. I do appreciate all of the information you have given me as every little bit helps me fit another piece of the puzzle in place. > From: igorrs@REDACTED > Date: Tue, 18 May 2010 03:08:52 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Mnesia questions > > On Mon, May 17, 2010 at 11:02 PM, Chris Hicks > wrote: > > I'm not sure I understand exactly what you are describing for the table you > > recommend, I think the dual PersonKey is what is throwing me off. What you > > are recommending is a separate item table, fragmented, where the primary key > > is the name of that separate (from the person table) table and the specific > > item ID. Then putting in another field that contains the ID of the > > associated person record in the person table and indexing on that. This way > > each item can be looked up singularly or as the group that belongs to that > > single player. Correct? > > What is the advantage to having the items spread out over multiple fragments > > as opposed to having them all located in a single table in the case of a > > bag? > > I was making some assumptions here (which may be wrong): > - That you are using fragmented tables anyway. > - That you are spreading your fragments across several nodes. > - That, if one of the nodes is temporarily unavailable, you prefer to > serve partial data to many "person"s than to serve nothing to some and > everything to some other. > > As for the advantages of using fragmented tables, I can only think of two now: > - Being able to spread your data across several servers. One can also > say that having data spread across different files will also improve > the fault tolerance of your system. > - Overcoming the 2 GB limit of dets (or not allowing dets files to > grow too big, for performance reasons). > > I should also point that I have both variants in production (using > bags and using the scheme I described above), both holding big amounts > of data (I can't say numbers, but it's more than what you described) > and both using dets (since the data doesn't fit in memory, even with > several servers). > If I could go back in time, I wouldn't be using bags. Although they > give me a little bit more performance (but again: all of this depends > on the operations you need to do) and save some space, I strongly > prefer to not have users with all of their data unavailable because > one server is down (it hasn't happened so far, but I already hate bags > ;-)). > > Best regards. > Igor. > > >> From: igorrs@REDACTED > >> Date: Mon, 17 May 2010 21:35:35 -0300 > >> To: silent_vendetta@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: Re: [erlang-questions] Mnesia questions > >> > >> To model your data, I think it's a good idea to first decide what are > >> the operations you're going to need. > >> > >> Also, I can't understand whether each item is specific of a person or > >> the same item can belong to more than one person. > >> > >> Supposing that each item belongs to only one person and that you need > >> to be able to: > >> - Retrieve all the items of a person. > >> - Retrieve a specific item of a person. > >> - Insert an item for a person. > >> > >> I think you would be fine with a table that has: > >> - {PersonKey, ItemKey} as the primary key. > >> - Another field for PersonKey. > >> - An index on the field above. > >> > >> This way, the items of the same user may be distributed among several > >> fragments, what is probably what you want. > >> > >> Using a bag with the person as the primary key, all the items for the > >> same user will be on the same fragment. Also, accessing a specific > >> item will be slower. > >> > >> Good luck. > >> Igor. > >> > >> On Mon, May 17, 2010 at 8:32 PM, Chris Hicks > >> wrote: > >> > > >> > First off thank you to everyone who responded publicly and privately. I > >> > do have a couple more specific ones about Mnesia and fragmented tables (yes, > >> > more of THOSE questions). > >> > 1) Well my first question is actually about tables in general. When one > >> > inserts/deletes a record/row from a table, is the whole table locked(didn't > >> > see this anywhere but could have missed it)? > >> > 2) I understand the general concept behind fragmented tables, and the > >> > answer to the above question may render part of this moot, but what are the > >> > performance pros/cons for using fragmented tables? I don't mean disk space > >> > but read/writes which occur mostly in transactions. Is each one any slower? > >> > Is there any advantage in parallelization? > >> > 2a)If a whole table is locked for an update does that just mean > >> > (hopefully) that the specific fragment of the larger table would only be > >> > locked, meaning a 50 fragment table could have 50 different locks/writes > >> > going? Of course if a whole table is not locked for an update that doesn't > >> > matter. > >> > One of the data requirements for my project is that one "person" record > >> > will be associated with possibly as many as 1000 "item" records. Lets take a > >> > possible person population of 10,000, meaning there could be as many as 10M > >> > item records. I want to keep each record small as I will be accessing > >> > different parts of a whole "object" at different times and will only need > >> > one or two small pieces of data at a time. One of the ways, and some of this > >> > is going to rely on the answers above, I was thinking of doing this was > >> > setting up a bag type table and associating all of those items (because they > >> > will be different) with the same key, the id of the person record. The one > >> > thing I'm wondering about, however, is some of these items might be in table > >> > fragment 1 and others in fragments 13, 27 and 42 (as far as I understand it) > >> > and I was wondering how much of a problem this would create as far as > >> > performance. > >> > Having each item with a different primary key, with each key held in a > >> > field in the person record, would mean I have a ton of queries if I want to > >> > get detailed info about each item and that doesn't work for me. Is there > >> > another approach that would be better that I am not thinking of? > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > ________________________________ > > The New Busy is not the too busy. Combine all your e-mail accounts with > > Hotmail. Get busy. > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 From erlang@REDACTED Tue May 18 11:06:12 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 May 2010 11:06:12 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> Message-ID: On Tue, May 18, 2010 at 5:25 AM, Eric Newhuis (personal) wrote: > The specific case I've had a number of times is when writing short filter functions that either pass through or modify something. > > I'd find myself almost forced to write the following: > > case find(...) of > ? ? ? ?error -> > ? ? ? ? ? ? ? ?{ok, Default}; > ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ?{ok, Value} > end. > > And the reconstruction of a small tuple bugged me. ?Your suggestion shortens this to something arguably easier to type. > > case X = find(...) of > ? ? ? ?error -> > ? ? ? ? ? ? ? ?{ok, Default}; > ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ?X > end. I'd write: case find(...) of error -> {ok, Default}; X -> X end. (( assuming I know that find returns error | {ok, Value} )) Save the compiler some work :-) /Joe > > ...without the apparent overhead of reconstruction. ?(Is it a moot point due to a clever compiler?) > > But I guess I am still concerned about suggesting this as a general programming idiom because it introduces a temporary variable. ?And as such one would need to study pre- and post-logic to avoid accidental use of the same variable name to name something different. > > I'd not have this concern if Erlang had a LET construct: > > let X = find(...) in > ? ? ? ?case X of > ? ? ? ? ? ? ? ?error -> > ? ? ? ? ? ? ? ? ? ? ? ?{ok, Default}; > ? ? ? ? ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ? ? ? ? ?X > ? ? ? ?end. > > Variable scope is important. ?Of course...absurdly... > > (fun() -> > ? ? ? ?case X=find(...) of > ? ? ? ? ? ? ? ?error -> > ? ? ? ? ? ? ? ? ? ? ? ?{ok, Default1}; > ? ? ? ? ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ? ? ? ? ?X > ? ? ? ?end > end)(), > (fun() -> > ? ? ? ?case X=obviously_different_fetch(...) of > ? ? ? ? ? ? ? ?error -> > ? ? ? ? ? ? ? ? ? ? ? ?{ok, Default2}; > ? ? ? ? ? ? ? ?{ok, Value} -> > ? ? ? ? ? ? ? ? ? ? ? ?X > ? ? ? ?end > end)(). > > But that is ridiculous. > > So apparently my idea isn't so great. ?And I also don't have a satisfying temporary-free and reconstruction-free solution. > > > On May 17, 2010, at 6:09 PM, Richard O'Keefe wrote: > >> >> On May 18, 2010, at 2:00 AM, Eric Newhuis (personal) wrote: >> >>> For the record, I might still disagree, so far. ?I'm not sure. ?Simply for argument's sake... >>> >>> The context in which this might be maximally useful is the following. >>> >>> case some_module:some_function(...) of >>> ? ? ?{some, pattern} -> _; >>> ? ? ?{some, other, pattern} -> _; >>> ? ? ?_ -> whatever >>> end. >> >> What is wrong with >> >> ? ?case X = some_module:some_function(...) >> ? ? ?of {some, pattern} -> X >> ? ? ? ; {some, other, pattern} -> X >> ? ? ? ; _ -> whatever >> ? ?end >>> >>> Note that I belong to the school of philosophy that suggests that the number of temporary variables should be minimized. >> >> By using the wild card, you have *NOT* minimised the number of temporary >> variables. ?All you have done is to carefully deprive your reader of any >> clues as to what it is about. >> >> >>> I don't understand why the above would be called wild-card abuse. >> >> Because the whole *point* of the wild-card is that each and >> every occurrence of "_" should represent a DIFFERENT variable. >> That's what it means in Prolog, Mercury, Strand88, Parlog, GHC, >> ML, Haskell, Clean, ... ?Every time, a different variable. >> But you are relying on the "_" in each arm of the case being >> the *SAME* variable. >> >> You are also creating great confusion. >> Suppose I have >> >> ? ? ? case foo() >> ? ? ? ? of {ping,_} -> _ >> ? ? ? ? ?; {_,pong} -> _ >> ? ? ? end >> >> The wild cards *following* the arrows are the *same* variable; >> what about the wild cards inside the patterns? ?If not, why not? >> >> What if I write >> ? ? ? case foo() of _ -> case bar() of _ -> _ end end >> Does this mean the same as >> ? ? ? case foo() of X -> case bar() of X -> X end end >> or ? ?case foo() of X -> case bar() of Y -> X end end >> or ? ?case foo() of X -> case bar() of Y -> Y end end >> or what? >> >> There's another point. ?In Erlang as it stands, _every_ >> variable _without exception_ must be visibly present at >> the point where it is bound. ?(Wild cards are no exceptions >> to this rule). ?You are introducing a new reading of "_" >> that violates this rule. ?If you want to think of the >> variable as bound in the pattern, write >> ? ? ? case Expr >> ? ? ? ? of X = Pattern when Guard -> X >> If you want to think of the variable as bound in the head, >> write >> ? ? ? case X = Expr >> ? ? ? ? of Pattern when Guard -> X >> In either case, the variable X is *visibly* bound. >> >>> It is clear from context that the wild-card represents something other than matching. >> >> Yes, but we already *have* something we can use for this purpose. >> Ordinary variable names. >> >> >>> We've seen this idea before. ?There are those grammars that expose variables whose value is whatever matched. >> >> I have no idea what you are referring to. ?Can you explain? >> >> "Exposing variables" is different from "invisibly binding anonymous >> variables". >> >>> >>> I suppose the following is where this great idea of mine might break down. >>> >>> case some_module:some_function(...) of >>> ? ? ?{some, pattern} -> >>> ? ? ? ? ? ? ?{encapsulated, _}; >>> ? ? ?{some, other, pattern} -> >>> ? ? ? ? ? ? ?{another, _, encapsulation} >>> end. >>> >>> Although I still don't have a problem with that. >> >> You may not. ?I do. >> >>> From context I know that the right hand side of the arrow isn't pattern matching. >> >> You as author may; your reader WILL have to work harder in reading >> to find out what the context *is*, especially if (as is often the >> case) the arrow is on the previous screen. >> >> This would be better as >> >> ? ? ? case X = some_module:some_function(...) >> ? ? ? ? of {some, pattern} -> >> ? ? ? ? ? ? ? ?{encapsulated, X} >> ? ? ? ? ?; {some, other, pattern} -> >> ? ? ? ? ? ? ? ?{another, X, encapsulation} >> ? ? ? end >> >> There is not one of your examples that would not be massively improved >> by introducing a named variable, even a literal X. >>> >>> I guess where readability might break down is in nesting: >>> >>> case some_module:some_function(...) of >>> ? ? ?{some, _, pattern} -> % _1 >>> ? ? ? ? ? ? ?case _ of -> ?% _2 >>> ? ? ? ? ? ? ? ? ? ? ?{some, great, pattern} -> >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?not_so_bad; >>> ? ? ? ? ? ? ? ? ? ? ?_ -> % _3 >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{_, Kind, _} = _, % _4, _5, _6 >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Kind >>> ? ? ? ? ? ? ?end >>> end. >>> >>> Although I can still read the above once I learn that underscore ('_') is context sensitive. >>> >>> _1 :: any() >>> _2 :: {some, any(), pattern} >>> _3 :: {some, any(), pattern}, not {some, great, pattern} >>> _4 :: some >>> _5 :: pattern >>> _6 :: _3 >> >> If I want to play at silly puzzles I do the Sudoku problem in the >> newspaper. ?If I'm reading a program, I do *NOT* enjoy pointless >> stumbling-blocks placed in my path to understanding. >> >> Let's rewrite your example. >> >> ? ? ? case some_module:some_function(...) >> ? ? ? ? of {some,Kind,pattern} -> >> ? ? ? ? ? ? ? ?case Kind >> ? ? ? ? ? ? ? ? ? of great -> not_so_bad >> ? ? ? ? ? ? ? ? ? ? _ ? ? -> Kind >> ? ? ? ? ? ? ? ?end >> ? ? ? end >> >> There is now *no* puzzle-solving for the reader. >> >> Let me raise my usual refrain: >> ? ? ? let's have a REAL example. >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From erlang@REDACTED Tue May 18 11:35:25 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 May 2010 11:35:25 +0200 Subject: [erlang-questions] Re: Defensive Programming In-Reply-To: References: <4BF06BEC.1070106@eonblast.com> <0d615095-3759-4e4e-9594-0bb75d56c986@l6g2000vbo.googlegroups.com> <4BF07BCF.7020309@eonblast.com> <4BF0E6D0.4010609@erlang-solutions.com> Message-ID: Programs crash either by design (it was *supposed* to crash) or by omission (you omitted to write code to stop it from crashing) Imagine dividing by zero. I write: X = A/B When I write this line of code I think to myself - "what happens if B is zero?" and "If B is is zero am I happy that the default recovery mechanisms take care of the error?" If the answer is yes - (ie I'm happy that the default mechanisms take care of the error) then I just write A/B. If the answer is no, then I must write defensive code: But writing the defensive code is *difficult* I can't write: X = case B of 0 -> .... do something else ... _ -> A/B end This would be logical nonsense, X has no value if B is 0 (you *could* write X = nAn (not a Number) but then you'd have to handle it at all points where X ( a number) was expected. So you'd have to write case B of 0-> do this; _ -> do something else end. This would break the entire structure of your code, a simple A/B in the middle of a sequence of statements who change the entire shape of the code foo() -> ... a, b, X = A/B, c, d. would change from a simple sequence to foo() -> ... a, b, case B of 0 -> ... something ... _ -> c, d end. Use divide a couple of times in your code and you'd soon get an impenetrable mess of codes. Letting the default recovery mechanism handle the error is almost always the right thing to do. If it is not the right thing to do, then you can 1) improve the default mechanism or 2) write your own error handling code. 1) is a good - since it will probably benefit other code as well (ie the default mechanism should be generic) (that's why gen_* things are popular - they try to "do the right thing" when you crash). 2) turns errors into "part of the problem" and as such they should be specified. Once they are specified, coded, and handled they no longer become errors they become defined and specified behavior. This argument is somewhat long .. the 0,5 second summary is "let it crash" /Joe On Mon, May 17, 2010 at 10:08 PM, Robert Virding wrote: > I was going to say that you should handle the error where it is most > sensible to do so but you said it better. :-) > > Robert > > 2010/5/17 Mazen Harake : >> I wrote this once... see if it makes sense: >> http://mazenharake.wordpress.com/2009/09/14/let-it-crash-the-right-way/ >> >> /Mazen >> >> >> On 17/05/2010 02:12, Henning Diedrich wrote: >>> >>> Thanks Steve, >>> >>> that question of "is it recoverable?" makes a lot of sense to clarify. I >>> am taking note. >>> >>> In my case I am writing a lib function for others to use, which is exactly >>> why I would like to give them more information on the error. >>> >>> Even though they might not be able to recover it, they should at least >>> know why it didn't work, instead of just getting a bad match for "ok = " >>> >>> I meanwhile conclude that the trimmest variant will be to care less and >>> demand that the user of the function test for the return value directly >>> handed over from the gen_tcp:send() call. Making it >>> >>> login(Name, Password) -> >>> >>> ? ? ?... >>> ? ? ? gen_tcp:send(Socket, term_to_binary(Str)). >>> >>> >>> >>> That looks the Erlangest an hands responsibility to the person who can >>> actually answer the 'recoverability question', the user of the lib. >>> >>> I realize I'd like to take away chances of frustration, puffing it up in >>> the process. >>> >>> Thanks for taking a look! >>> Henning >>> >>> >>> >>> Steve Davis wrote: >>>> >>>> Hi Henning, >>>> >>>> This isn't intended to be any kind of definite answer, and I'm >>>> interested to see how others respond. >>>> >>>> In my case how I deal with this question is that I'd probably ask >>>> myself: >>>> >>>> *Is this exceptional case _recoverable_,? e.g. in the case of the tcp >>>> send, you could try to resend the data, so the first idiom may do it. >>>> >>>> If it's not recoverable, I'd just let it crash out. In fact I would >>>> likely force it to crash out using the second idiom. >>>> >>>> If I want to get fancy with error handling, I find that try..catch at >>>> the top level is usually enough. >>>> >>>> A further aside to this is that a certain Mr Armstrong once introduced >>>> me to an interesting "transactional" idiom also, namely: >>>> >>>> try begin >>>> ... >>>> end catch >>>> ... >>>> >>>> Regards, >>>> >>>> /s >>>> >>>> On May 16, 5:04 pm, Henning Diedrich wrote: >>>>> >>>>> Hi list, >>>>> >>>>> I have a question about the Erlang way. >>>>> >>>>> I realize I keep programming defensive using exceptions/error calls to >>>>> *label failure* more precisely. >>>>> >>>>> E.g. ( - PRESUMABLY WRONG - ) >>>>> >>>>> login(Name, Password) -> >>>>> >>>>> ? ? ? ... >>>>> ? ? ? ?case >>>>> ? ? ? ? ? ? ? ?gen_tcp:send(Socket, term_to_binary(Str)) of >>>>> ? ? ? ? ? ? ? ? ? ? ok -> ok; >>>>> ? ? ? ? ? ? ? ? ? ? _ -> erlang:error(*sending_failed, {Str }*) >>>>> ? ? ? end >>>>> ? ? ? ... >>>>> >>>>> Instead of simply >>>>> >>>>> login(Name, Password) -> >>>>> >>>>> ? ? ? ... >>>>> ? ? ? ?ok = gen_tcp:send(Socket, term_to_binary(Str)), >>>>> ? ? ? ... >>>>> >>>>> It feels wrong in Erlang and I wonder if I am missing out on something. >>>>> >>>>> If somebody has a pointer to read up on this, thanks in advance, >>>>> >>>>> Henning >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Erlang Programming" group. >>>>> To post to this group, send email to >>>>> erlang-programming@REDACTED >>>>> To unsubscribe from this group, send email to >>>>> erlang-programming+unsubscribe@REDACTED >>>>> For more options, visit this group >>>>> athttp://groups.google.com/group/erlang-programming?hl=en. >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>> >> >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG >> SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From erlang@REDACTED Tue May 18 11:46:41 2010 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 May 2010 11:46:41 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: There's another problem. What abstractions do we use to express concurrency in. We can use spawn send, receive and so on to model any of the concurrency patterns I mentioned. But these are rather low level. pmap and pipe are much higher level. pmap works just like map only the arguments are evaluated in parallel. now pmap(fun(X) -> 2*X end, List) doesn't make much sense on a multicore the overhead of splitting and joining the arguments outweighs the benefit. pmap(fun(X) -> compile(X) end, List) might make sense - since the work done is large compared to the setup overhead. But wait a moment, do we want to pmap over all elements in a list. pmap on a thousand element list will create 1000 parallel processes. On a quad core it might would make sence to break the list into 4 chunks (not 1000) - or 8 chunks on a 8-core. There's no point in expressing the concurrency in an algorithm if the hardware can't do it anyway (or perhaps there is?) Seems like we also need a topology layer te separate the pmap abstraction from the characteristics of the underlying hardware, taking into account physical characteristics of the system. We might even need a micro-scheduler, just for managing a pmap operation. Tricky stuff /Joe On Sat, May 15, 2010 at 7:46 PM, Jay Nelson wrote: > In an earlier thread I alluded to not understanding how to express > concurrency patterns. ?My belief was that they are more complicated to > categorize and differentiate than design patterns of OO because they cross > the architecture rather than just a single software module. ?I wasn't sure > what elements were necessary to describe them and think that is starting to > show in the discussion on this thread. I think it will take a little > analysis and negotiation to arrive at a reasonable description. > > Thus far, this thread has mainly enumerated specific instances of > architectures without looking at the attributes and features of the entire > problem space with a goal of identifying the measures which would > differentiate approaches. ?A proper analysis should result in some as yet > uncommon or undiscovered concurrency patterns by filling in missing examples > on a diagram of possibilities. > > To make progress we need to look at the range of features needed to describe > the domain of concurrency patterns. ?Here are some key attributes of a > concurrent architecture that I can think of offhand, along with measures of > each that could be used to classify them (considering only process-based > concurrency), the main point being different ways of describing the > processes that participate in a pattern: > > Number of processes > ? - One => ?all OO design patterns fall in here > ? - Few => typically task oriented processes with dedicated functionality > ? - Dozens => typically partitioned systems with worker pools > ? - Many => event driven spawning or grid-like computations > ? - Dynamic => distributed hashes, communal participatory computation (SETI) > > Lifetime of processes > ? - Static/Infinite => dedicated services essential to working system > ? - Enduring => self-recovering stable functions (reconnect on disconnect), > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?resource pool workers > ? - Temporary => services needed on demand, caching > ? - Ephemeral => reactive workers or task-oriented computation > ? - Dormant => resource conserving (hibernation) > > Scheduling of processes > ? - Once => static service > ? - Periodic => cron-like services, Comet / AJAX polling > ? - On Demand => event-driven services > > Static Organization of processes > ? - Network computation topologies => hypercube, star, mesh, ring, etc. > ? - Functional relationships => supervisors, serial servers, pub/sub, pool > ? - Partitioning => isolation of failure, allocation of resources > > Dynamic Organization of processes > ? - Load adaptive => efficient resource allocating services > ? - Demand driven => swarm computing > ? - Migratory => mobile agents, resource discovery, continuation patterns > ? - Coordinated => failover / takeover, command and control, leader election > ? - Participatory => distributed hash, SETI > > Data flow through processes > ? - Static => networked dataflow, traditional partitioned > ? - Affinity-based => data flocking for efficient computation > ? - Routed / Queue distributed => pub/sub, worker pools > ? - Activation propagation => traditional dataflow, pipeline > ? - Central distributed => serial server, rendezvous > ? - Scatter / Gather => map reduce, grid > ? - Network overlay => SIMD grid > > Location of processes > ? - Static => traditional concurrency > ? - Dynamic => mobile agents, adaptive load systems > ? - Admin directed => command and control concurrency > ? - Resource associated => adaptive load, efficient data computation > > Process-Mapped Resource Access > ? - Continuous => traditional > ? - Repeated static => pool workers > ? - Repeated dynamic => event driven, load adaptive > ? - One-shot => caching, dataflow, tcp proxy > > As you can see, the OO patterns eliminate a few dimensions of the range of > possibilities and therefore are presumably easier to describe. > > Not all of these features are necessary simultaneously to classify the > entire domain (as seen by some of my overlapping descriptions), arguments > exist for selective sets of them or combinations. ?I think that because a > concurrent system is more complex and dynamic, it is necessary to describe > an architecture with both static attributes (typically called "the > architecture") and dynamic attributes (process lifecycle, dataflow) over > time. > > I think this is a highly relevant thread, and an important topic for the > Concurrency Oriented Programming Language (COPL) community to come to > agreement on. ?Doing so will advance software and training at the pace of > the multi-core options that are becoming available. ?I think this outline > could form the basis of a collaborative wiki to discover a better > classification hierarchy. > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From geoffrey.biggs@REDACTED Tue May 18 11:54:41 2010 From: geoffrey.biggs@REDACTED (Geoffrey Biggs) Date: Tue, 18 May 2010 18:54:41 +0900 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: <4BF263E1.4060006@aist.go.jp> On 18/05/10 18:46, Joe Armstrong wrote: > There's no point in expressing the concurrency in an algorithm if the > hardware can't do it anyway > (or perhaps there is?) I believe there is. It may not be directly applicable to hardware now, but it has a high chance of being applicable at some point in the future. Just make sure it's *implementable* now, even if we don't get all the benefits that having native hardware support would give, like true concurrency. The catch here is that your example of pmap breaking into different sizes depending on the number of cores is the anti-thesis of what a pattern should be. It's specifying implementation details. I think that the patterns should be more abstract than "the task is split up 4 times" - a better description is "the task is split up n times," with n determined by the pattern user for their specific needs - hopefully based on some benchmarking. Geoff From jesper.louis.andersen@REDACTED Tue May 18 12:39:33 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 May 2010 12:39:33 +0200 Subject: [erlang-questions] log_mf_h and rb do not handle terms over 65k In-Reply-To: References: Message-ID: On Mon, May 17, 2010 at 8:15 PM, Garret Smith wrote: > I was logging a couple very large terms using > error_logger:info_report and log_mf_h during application start, > and was always unable to read most of the first log file > generated. ?This was mystifying me for a while until I finally > traced the source. > > log_mf_h and rb use a 2-byte length indicator for the term they > are about to read/write, therefore they cannot handle terms > over 65,536 bytes in length. ?See handle_event/2 in log_mf_h > and read_report/1 in rb Yes, etorrent has the same problem for some of its processes. I reported this little problem a couple of years ago, and even did the backwards-incompatible patch against R11b-5 to fix it: http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187 -- J. From wiener.guy@REDACTED Tue May 18 14:29:07 2010 From: wiener.guy@REDACTED (Guy Wiener) Date: Tue, 18 May 2010 15:29:07 +0300 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: <4BF263E1.4060006@aist.go.jp> References: <4BF263E1.4060006@aist.go.jp> Message-ID: IMHO we need to distinguish between two things: Dependencies and Resources. Dependencies in a pattern describe the process topology under optimal circumstances. For example, in pmap, there is no dependency between the processes. In a pipe, each process depends on the previous. In a star, processes depend on a central, long-running process, etc. Resources, as Joe and Geoff mentioned, describe what is available to the application - Namely, the number of processors and their network topology. The actual instance of the application is the combination: Assigning available resources according to the dependencies. Sometime there are un-used processes (for example, if the pipe is shorter then the available processors). Sometimes the actual assignment is less then the optimal (for example, if there are more independent processes then available processors). Guy. On Tue, May 18, 2010 at 12:54 PM, Geoffrey Biggs wrote: > On 18/05/10 18:46, Joe Armstrong wrote: > > There's no point in expressing the concurrency in an algorithm if the > > hardware can't do it anyway > > (or perhaps there is?) > > I believe there is. It may not be directly applicable to hardware now, > but it has a high chance of being applicable at some point in the > future. Just make sure it's *implementable* now, even if we don't get > all the benefits that having native hardware support would give, like > true concurrency. > > The catch here is that your example of pmap breaking into different > sizes depending on the number of cores is the anti-thesis of what a > pattern should be. It's specifying implementation details. I think that > the patterns should be more abstract than "the task is split up 4 times" > - a better description is "the task is split up n times," with n > determined by the pattern user for their specific needs - hopefully > based on some benchmarking. > > Geoff > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From christophe.romain@REDACTED Tue May 18 16:13:19 2010 From: christophe.romain@REDACTED (Christophe Romain) Date: Tue, 18 May 2010 16:13:19 +0200 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: Message-ID: <20100518141319.GA3844@localhost> look at CEAN http://cean.process-one.net/ usage is described here http://cean.process-one.net/doc/ you can search list install update.... The problem is that this has been done on spare time, it's quite outdated now (R12B-4 based), and spare time is more than seldom. I have R13B04 repository working with lots of code upgrade, but need to update the official site and make automatic daily update running on the site. Both erlware and cean have pros/cons depending of what you need. Both are really usable systems. Anyway, Erlang really lack of an "official" and "standard" packaging centralized/mirrored solution. That's what i tryed to do with CEAN. let's take an example on yaws packet: http://cean.process-one.net/packages/index.yaws?action=detail&name=yaws each package contains all needed metadata. CEAN automatically get project sources and format it the OTP way so all packages should be ready to use. It also try to automatically guess status of packages (see http://cean.process-one.net/status/) CEAN also does not depend on an erlang installation. you can just download CEAN bootstrap (3Mb) and you have an erlang shell in 1 minute without any root privilege. We're talking about erlang packaging for more that 4 years, i'd really like the community to have a concencus on what we need and how we do/maintain it. spreading in many systems is not good, except if we have gateways (i'm almost certain we could write script that exports cean's package to erlware and import erlware contribution as cean package) By the way, i really liked Joe's idea on elib, and i think it can be good practice. any thoughts ? From garret.smith@REDACTED Tue May 18 16:53:21 2010 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 18 May 2010 07:53:21 -0700 Subject: [erlang-questions] log_mf_h and rb do not handle terms over 65k In-Reply-To: References: Message-ID: Would the PTB be more willing to accept a backwards compatible patch that either truncates the term or replaces it with a "Term too big" message? I would be happy to create one if there is a good chance of it being accepted. -Garret Smith On Tue, May 18, 2010 at 3:39 AM, Jesper Louis Andersen wrote: > On Mon, May 17, 2010 at 8:15 PM, Garret Smith wrote: >> I was logging a couple very large terms using >> error_logger:info_report and log_mf_h during application start, >> and was always unable to read most of the first log file >> generated. ?This was mystifying me for a while until I finally >> traced the source. >> >> log_mf_h and rb use a 2-byte length indicator for the term they >> are about to read/write, therefore they cannot handle terms >> over 65,536 bytes in length. ?See handle_event/2 in log_mf_h >> and read_report/1 in rb > > Yes, etorrent has the same problem for some of its processes. I > reported this little problem a couple of years ago, and even did the > backwards-incompatible patch against R11b-5 to fix it: > > http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187 > > -- > J. > From tuncer.ayaz@REDACTED Tue May 18 17:20:55 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 18 May 2010 17:20:55 +0200 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: <20100518141319.GA3844@localhost> References: <20100518141319.GA3844@localhost> Message-ID: For completeness: http://github.com/JacobVorreuter/epm http://github.com/JacobVorreuter/sutro From zeno490@REDACTED Tue May 18 17:36:05 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Tue, 18 May 2010 11:36:05 -0400 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: IMO, it is premature to talk about an erlang library management system when the risk of module name collision is so high. We'd have to require/enforce a package prefix in the form of 'packagename_' to all modules of a package. That is until erlang has an approved package/namespace system (the one currently implemented is experimental and discussed in another thread atm). Before we can run, we must first learn to walk. Nicholas On Tue, May 18, 2010 at 11:20 AM, Tuncer Ayaz wrote: > For completeness: > http://github.com/JacobVorreuter/epm > http://github.com/JacobVorreuter/sutro > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From seanhern@REDACTED Tue May 18 18:11:45 2010 From: seanhern@REDACTED (Sean Hernandez) Date: Tue, 18 May 2010 11:11:45 -0500 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: <4BF263E1.4060006@aist.go.jp> Message-ID: I believe more comunications patterns can be found in the following technical report, The Landscape of Parallel Computing Research: A View from Berkeley http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html On Tue, May 18, 2010 at 7:29 AM, Guy Wiener wrote: > IMHO we need to distinguish between two things: Dependencies and Resources. > > Dependencies in a pattern describe the process topology under optimal > circumstances. For example, in pmap, there is no dependency between the > processes. In a pipe, each process depends on the previous. In a star, > processes depend on a central, long-running process, etc. > > Resources, as Joe and Geoff mentioned, describe what is available to the > application - Namely, the number of processors and their network topology. > > The actual instance of the application is the combination: Assigning > available resources according to the dependencies. Sometime there are > un-used processes (for example, if the pipe is shorter then the available > processors). Sometimes the actual assignment is less then the optimal (for > example, if there are more independent processes then available > processors). > > Guy. > > On Tue, May 18, 2010 at 12:54 PM, Geoffrey Biggs > wrote: > > > On 18/05/10 18:46, Joe Armstrong wrote: > > > There's no point in expressing the concurrency in an algorithm if the > > > hardware can't do it anyway > > > (or perhaps there is?) > > > > I believe there is. It may not be directly applicable to hardware now, > > but it has a high chance of being applicable at some point in the > > future. Just make sure it's *implementable* now, even if we don't get > > all the benefits that having native hardware support would give, like > > true concurrency. > > > > The catch here is that your example of pmap breaking into different > > sizes depending on the number of cores is the anti-thesis of what a > > pattern should be. It's specifying implementation details. I think that > > the patterns should be more abstract than "the task is split up 4 times" > > - a better description is "the task is split up n times," with n > > determined by the pattern user for their specific needs - hopefully > > based on some benchmarking. > > > > Geoff > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > From jesper.louis.andersen@REDACTED Tue May 18 18:18:34 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 May 2010 18:18:34 +0200 Subject: [erlang-questions] log_mf_h and rb do not handle terms over 65k In-Reply-To: References: Message-ID: On Tue, May 18, 2010 at 4:53 PM, Garret Smith wrote: > Would the PTB be more willing to accept a backwards compatible patch that > either truncates the term or replaces it with a "Term too big" message? I would definitely recommend replacing it with a term-too-big message. As it stands at the moment, the bug means that the rest of your log becomes unusable because it can't be read correctly. That is a log worse than missing out on a single crash report. -- J. From mihai@REDACTED Tue May 18 18:42:44 2010 From: mihai@REDACTED (Mihai Balea) Date: Tue, 18 May 2010 12:42:44 -0400 Subject: Who is killing my process? Message-ID: <6077C19D-3640-40F0-9DEE-D8CEB2462294@hates.ms> Hi all, In one of my systems I have a bunch of worker processes that are spawned on demand using a simple_one_for_one supervisor. Occasionally, one of the workers dies and the supervisor reports the reason as "killed". To my knowledge, that means some external entity is sending that process a kill signal, the problem is I am not able to figure out where it comes from. There is no place in my code that calls exit(Pid, kill). Tracing on the supervisor's shutdown functions show that they are not called. Is there any way one can tell who is killing the process? Thanks, Mihai From mail@REDACTED Tue May 18 18:49:43 2010 From: mail@REDACTED (Tim Fletcher) Date: Tue, 18 May 2010 09:49:43 -0700 (PDT) Subject: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: <7031fc78-70ac-4ce4-ab6e-1864cb17df28@f14g2000vbn.googlegroups.com> > IMO, it is premature to talk about an erlang library management system when > the risk of module name collision is so high. We'd have to require/enforce a > package prefix in the form of 'packagename_' to all modules of a package. Or the library management system could throw an error when trying to install a library with module names that already exist. Or you could allow libraries to be installed without restriction, and use a well defined ordering when loading modules (i.e., alphabetical, or by installation date). In principle I think the tools should be able to deal with things like this without forcing everyone to change their code. Tim From bob@REDACTED Tue May 18 19:01:20 2010 From: bob@REDACTED (Bob Ippolito) Date: Tue, 18 May 2010 10:01:20 -0700 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: On Tue, May 18, 2010 at 8:36 AM, Nicholas Frechette wrote: > IMO, it is premature to talk about an erlang library management system when > the risk of module name collision is so high. We'd have to require/enforce a > package prefix in the form of 'packagename_' to all modules of a package. Many (most?) packages already do this, so it's not really a problem. > That is until erlang has an approved package/namespace system (the one > currently implemented is experimental and discussed in another thread atm). There's plenty other global resources you can run into that aren't just module names, e.g. processes or applications. Picking a namespace syntax doesn't fix that. Also if you don't install packages system-wide, you're even less likely to have a problem. You could use something like Basho's rebar to manage application installs with dependencies. -bob From dale@REDACTED Tue May 18 19:08:24 2010 From: dale@REDACTED (Dale Harvey) Date: Tue, 18 May 2010 18:08:24 +0100 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: On 18 May 2010 18:01, Bob Ippolito wrote: > On Tue, May 18, 2010 at 8:36 AM, Nicholas Frechette > wrote: > > IMO, it is premature to talk about an erlang library management system > when > > the risk of module name collision is so high. We'd have to > require/enforce a > > package prefix in the form of 'packagename_' to all modules of a package. > > Many (most?) packages already do this, so it's not really a problem. > > > That is until erlang has an approved package/namespace system (the one > > currently implemented is experimental and discussed in another thread > atm). > > There's plenty other global resources you can run into that aren't > just module names, e.g. processes or applications. Picking a namespace > syntax doesn't fix that. > > and the erlang "application" infrastructure already warns(errors?) against conflicts in module names / global processes etc. As long as the library management system was based around .app files, doesnt seem like there would be a problem > Also if you don't install packages system-wide, you're even less > likely to have a problem. You could use something like Basho's rebar > to manage application installs with dependencies. > > -bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rapsey@REDACTED Tue May 18 19:13:26 2010 From: rapsey@REDACTED (Rapsey) Date: Tue, 18 May 2010 19:13:26 +0200 Subject: [erlang-questions] Who is killing my process? In-Reply-To: <6077C19D-3640-40F0-9DEE-D8CEB2462294@hates.ms> References: <6077C19D-3640-40F0-9DEE-D8CEB2462294@hates.ms> Message-ID: Are you updating your code? Processes are killed when they're running old code and it gets updated. Sergej On Tue, May 18, 2010 at 6:42 PM, Mihai Balea wrote: > Hi all, > > In one of my systems I have a bunch of worker processes that are spawned on > demand using a simple_one_for_one supervisor. Occasionally, one of the > workers dies and the supervisor reports the reason as "killed". To my > knowledge, that means some external entity is sending that process a kill > signal, the problem is I am not able to figure out where it comes from. > There is no place in my code that calls exit(Pid, kill). Tracing on the > supervisor's shutdown functions show that they are not called. Is there any > way one can tell who is killing the process? > > Thanks, > Mihai > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From anthonym@REDACTED Tue May 18 19:11:18 2010 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 18 May 2010 10:11:18 -0700 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: <20100518171118.GD2427@alumni.caltech.edu> Also for completeness http://code.google.com/p/fwtemplates/wiki/FwTemplateErlangWalkthrough Manage erlang "applications" as system rpms/debs with hooks for hot code loading during a 'yum upgrade' or 'apt-get' -Anthony On Tue, May 18, 2010 at 05:20:55PM +0200, Tuncer Ayaz wrote: > For completeness: > http://github.com/JacobVorreuter/epm > http://github.com/JacobVorreuter/sutro > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From comptekki@REDACTED Tue May 18 19:34:29 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 11:34:29 -0600 Subject: erl command line options Message-ID: I've looked in the otp-system-doc and around on erlang.org. Where can I find info on the command line options when starting "erl". I even tried erl -help and erl --help - they just go in to the shell. thx, -wes From dale@REDACTED Tue May 18 19:43:21 2010 From: dale@REDACTED (Dale Harvey) Date: Tue, 18 May 2010 18:43:21 +0100 Subject: [erlang-questions] erl command line options In-Reply-To: References: Message-ID: yeh they arent exactly easy to find http://erldocs.com/otp_src_R13B/erts/init.html#flags On 18 May 2010 18:34, Wes James wrote: > I've looked in the otp-system-doc and around on erlang.org. Where can > I find info on the command line options when starting "erl". I even > tried erl -help and erl --help - they just go in to the shell. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dale@REDACTED Tue May 18 19:44:20 2010 From: dale@REDACTED (Dale Harvey) Date: Tue, 18 May 2010 18:44:20 +0100 Subject: [erlang-questions] erl command line options In-Reply-To: References: Message-ID: er sorry, old version - http://erldocs.com/R13B04/erts/init.html?search=init&i=0#flags On 18 May 2010 18:43, Dale Harvey wrote: > yeh they arent exactly easy to find > > http://erldocs.com/otp_src_R13B/erts/init.html#flags > > > On 18 May 2010 18:34, Wes James wrote: > >> I've looked in the otp-system-doc and around on erlang.org. Where can >> I find info on the command line options when starting "erl". I even >> tried erl -help and erl --help - they just go in to the shell. >> >> thx, >> >> -wes >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From comptekki@REDACTED Tue May 18 19:53:44 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 11:53:44 -0600 Subject: erl command line options In-Reply-To: References: Message-ID: On Tue, May 18, 2010 at 11:34 AM, Wes James wrote: > I've looked in the otp-system-doc and around on erlang.org. ?Where can > I find info on the command line options when starting "erl". ?I even > tried erl -help and erl --help - they just go in to the shell. Thanks all for links. -wes From zeno490@REDACTED Tue May 18 19:56:32 2010 From: zeno490@REDACTED (Nicholas Frechette) Date: Tue, 18 May 2010 13:56:32 -0400 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: <20100518141319.GA3844@localhost> Message-ID: This is going to get quite painful, quite quickly. Whenever you'll be the first programmer to use two packages together that nobody used together before and a module, or registered process name clashes with one or the other (or with your code), you'll be left with a question of which to fix? Both (all three)? Won't this lead to packages not already prepending a prefix to everything to release a number of patches to 'fix' compatibility issues with other packages? The package system will not be able to fix/catch all. Yes it'll be able to warn you: don't do this, it conflicts but you'll be stuck fixing things yourself. In reality, there should be an encapsulation primitive in the language (like namespaces, for modules and registered processes and other global ressources). Without a language primitive, we'll be left with tight naming guidelines one has to follow to publish a package. IMO, an unnecessary hurdle. In fact, I'm sure that at ericsson, internally they must have strict naming guidelines in place to prevent clashes in case an app would be shared. While this might work at a company scale, it won't at a scale comparable to ruby gems. A feature like that is quite important to help code sharing and an erlang library sharing app would be a great justification to implement it. On Tue, May 18, 2010 at 1:08 PM, Dale Harvey wrote: > > > On 18 May 2010 18:01, Bob Ippolito wrote: > >> On Tue, May 18, 2010 at 8:36 AM, Nicholas Frechette >> wrote: >> > IMO, it is premature to talk about an erlang library management system >> when >> > the risk of module name collision is so high. We'd have to >> require/enforce a >> > package prefix in the form of 'packagename_' to all modules of a >> package. >> >> Many (most?) packages already do this, so it's not really a problem. >> >> > That is until erlang has an approved package/namespace system (the one >> > currently implemented is experimental and discussed in another thread >> atm). >> >> There's plenty other global resources you can run into that aren't >> just module names, e.g. processes or applications. Picking a namespace >> syntax doesn't fix that. >> >> > and the erlang "application" infrastructure already warns(errors?) against > conflicts in module names / global processes etc. > > As long as the library management system was based around .app files, > doesnt seem like there would be a problem > > >> Also if you don't install packages system-wide, you're even less >> likely to have a problem. You could use something like Basho's rebar >> to manage application installs with dependencies. >> >> -bob >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From brendan.doyle@REDACTED Tue May 18 19:40:07 2010 From: brendan.doyle@REDACTED (Brendan Doyle) Date: Tue, 18 May 2010 12:40:07 -0500 Subject: [erlang-questions] erl command line options In-Reply-To: References: Message-ID: http://erlang.org/doc/man/erl.html Brendan Doyle Manager, Application Development Epic Advertising - New York, Toronto, San Francisco, London www.EpicAdvertising.com 60 Columbia Way, Suite 310 Markham, ON L3R 0C9 (905) 946-0300 x.2358 work (647) 885-7159 mobile (888) 666-3120 fax brendan.doyle@REDACTED On 2010-05-18, at 1:34 PM, Wes James wrote: > I've looked in the otp-system-doc and around on erlang.org. Where can > I find info on the command line options when starting "erl". I even > tried erl -help and erl --help - they just go in to the shell. > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From erlangy@REDACTED Tue May 18 20:18:00 2010 From: erlangy@REDACTED (Michael) Date: Tue, 18 May 2010 11:18:00 -0700 Subject: [erlang-questions] Re: erl command line options In-Reply-To: References: Message-ID: <20100518181758.GC4020@delora.autosys.us> On Tue, May 18, 2010 at 11:53:44AM -0600, Wes James wrote: > On Tue, May 18, 2010 at 11:34 AM, Wes James wrote: > > I've looked in the otp-system-doc and around on erlang.org. ?Where can > > I find info on the command line options when starting "erl". ?I even > > tried erl -help and erl --help - they just go in to the shell. > > > Thanks all for links. > > -wes > ________________________________________________________________ and ... $ erl -man erl (presuming you installed the man files) ~Michael From kenneth.lundin@REDACTED Tue May 18 20:55:43 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 18 May 2010 20:55:43 +0200 Subject: [erlang-questions] log_mf_h and rb do not handle terms over 65k In-Reply-To: References: Message-ID: Yes, a backwards compatible patch which truncates the term or replaces it with "term too big" has very good chances to be accepted. Provided of course that it is submitted according to the instructions on Github. /Kenneth, Erlang/OTP Ericsson On Tue, May 18, 2010 at 4:53 PM, Garret Smith wrote: > Would the PTB be more willing to accept a backwards compatible patch that > either truncates the term or replaces it with a "Term too big" message? > > I would be happy to create one if there is a good chance of it being accepted. > > -Garret Smith > > On Tue, May 18, 2010 at 3:39 AM, Jesper Louis Andersen > wrote: >> On Mon, May 17, 2010 at 8:15 PM, Garret Smith wrote: >>> I was logging a couple very large terms using >>> error_logger:info_report and log_mf_h during application start, >>> and was always unable to read most of the first log file >>> generated. ?This was mystifying me for a while until I finally >>> traced the source. >>> >>> log_mf_h and rb use a 2-byte length indicator for the term they >>> are about to read/write, therefore they cannot handle terms >>> over 65,536 bytes in length. ?See handle_event/2 in log_mf_h >>> and read_report/1 in rb >> >> Yes, etorrent has the same problem for some of its processes. I >> reported this little problem a couple of years ago, and even did the >> backwards-incompatible patch against R11b-5 to fix it: >> >> http://www.erlang.org/cgi-bin/ezmlm-cgi/3/187 >> >> -- >> J. >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Tue May 18 22:13:10 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 14:13:10 -0600 Subject: erl startup Message-ID: I am trying to get yaws to start a module with: yaws -r my_module --mnesiadir /path/to/mnesiadata yaws is a shell script that builds this in to: erl +K true -pa /usr/local/lib/yaws/ebin -run yaws -run my_mod -mnesia dir '"/Users/zulu/0erl/prt"' -run mnesia start mnesia starts up fine, but my mod does not. After getting yaws started up I can then do my_mod:start() and it starts fine. According to the command line options -run should run my_mod:start() but it doesn't seem to do this, even if I copy and paste the line above (should be same anyway). Why doesn't -run my_mod fire a my_mod:start()? thx, -wes From timuckun@REDACTED Wed May 19 00:33:35 2010 From: timuckun@REDACTED (Tim Uckun) Date: Wed, 19 May 2010 10:33:35 +1200 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: <20100518141319.GA3844@localhost> References: <20100518141319.GA3844@localhost> Message-ID: > > We're talking about erlang packaging for more that 4 years, i'd really > like the community to have a concencus on what we need and how we > do/maintain it. spreading in many systems is not good, except if we have > gateways (i'm almost certain we could write script that exports cean's > package to erlware and import erlware contribution as cean package) > By the way, i really liked Joe's idea on elib, and i think it can be good > practice. I didn't mean to stir up a hornets nest :). As an outsider my only useful input would be that IMHO it would be a very good idea to have a system like gems. The python community also resisted a system for a long time but eventually settled on eggs and easyinstall. I don't know what any ruby programmer would do without gems and I don't know what any perl programmers would to without CPAN. It's just unthinkable. From rvirding@REDACTED Wed May 19 02:39:42 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 19 May 2010 02:39:42 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On 18 May 2010 11:46, Joe Armstrong wrote: > ... > > There's no point in expressing the concurrency in an algorithm if the > hardware can't do it anyway > (or perhaps there is?) Yes, there is. If the algorithm is best expressed in a concurrent way then you should definitely do so, even if the hardware can't do it. You are saying what you mean. Or it can be a hint to the implementation, for example using pmap instead of map means that you feel that the arguments can be evaluated in parallel, it is then up to the implementation/hardware to do it if possible. Robert From mihai@REDACTED Wed May 19 02:40:36 2010 From: mihai@REDACTED (Mihai Balea) Date: Tue, 18 May 2010 20:40:36 -0400 Subject: [erlang-questions] Who is killing my process? In-Reply-To: References: <6077C19D-3640-40F0-9DEE-D8CEB2462294@hates.ms> Message-ID: On May 18, 2010, at 1:13 PM, Rapsey wrote: > Are you updating your code? Processes are killed when they're running old > code and it gets updated. Yes, it turned out that was the case...a bunch of workers would be spawned for a specific job, and workers would load the latest version of the job callback module. If a worker had a chance to start working while another was still initializing, it would result in a code purge from under the first worker. Anyway, simply ensuring the code is loaded once before the workers are spawned solved the problem. Thanks, Mihai From ginahagg@REDACTED Wed May 19 02:58:10 2010 From: ginahagg@REDACTED (Gina Hagg) Date: Tue, 18 May 2010 17:58:10 -0700 Subject: default dialyzer plt file Message-ID: Can i transfer my default dialyzer.plt file from my local machine to another machine using the same erlang lib version, or do i have to build one for each machine locally. From comptekki@REDACTED Wed May 19 03:11:51 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 19:11:51 -0600 Subject: erl startup In-Reply-To: References: Message-ID: On Tue, May 18, 2010 at 2:13 PM, Wes James wrote: > I am trying to get yaws to start a module with: > > yaws -r my_module --mnesiadir /path/to/mnesiadata > > yaws is a shell script that builds this in to: > > erl +K true -pa /usr/local/lib/yaws/ebin ? ? ?-run yaws ? ? -run > my_mod ? -mnesia dir '"/Users/zulu/0erl/prt"' -run mnesia start > > mnesia starts up fine, but my mod does not. After getting yaws started > up I can then do my_mod:start() and it starts fine. ?According to the > command line options -run should run my_mod:start() ?but it doesn't > seem to do this, even if I copy and paste the line above (should be > same anyway). > > Why doesn't -run my_mod fire a my_mod:start()? Well after starting a new shell and starting things fresh I am finally getting the my_mod:start() to work: start() -> Id_Table=ets:new(start_id_table, []), io:format("~w~n", [Id_Table]), {ok, TRef}=timer:apply_interval(1000*60*5, prt, getc, []), % 1000 milliseconds * 60 * 5 = 5 minutes ets:insert(Id_Table, {id, TRef}), Id_Table. I see the output from the io:format above but the timer does not seem to be running after erlang starts up. I'll keep poking to see what's going on. -wes From ok@REDACTED Wed May 19 04:41:34 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 19 May 2010 14:41:34 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4BF203E1.9070905@eonblast.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <4BF1D8DD.8080000@eonblast.com> <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> <4BF203E1.9070905@eonblast.com> Message-ID: <760424A5-1E77-4A3B-85EE-0168B23FA48F@cs.otago.ac.nz> On May 18, 2010, at 3:05 PM, Henning Diedrich wrote: > Richard, righteous, > > I was just curious about the defense you'd mount to keep ~ for the > frame syntax. > > Where the first thing that hit my uneducated I was how the use of ~ > in it was so completely special --- as some other symbol uses in > Erlang are --- that it, well, may be an excellent fit. Why starting > to get conventional. A language proudly flying =< could also have ~ > to mean 'something like' while really, yes, significantly different > from =. I am baffled by this "so completely special" claim. The frames report goes to some trouble to point out that this use of ~ for maplets is *NOT* innovative. It was a straight steal from Xerox PARC. The Mesa programming language had an equivalent of 'make' called C/Mesa. For Mesa's successor Cedar, this was replaced by Eric Schimdt's "System Modeller", which took care of - versions - building - distribution using a "system model" (think "Makefile on steroids") which was nothing other than a pure lazy functional program for building systems. (This is the same Eric Schmidt who became CEO of Google.) A lovely idea, and anyone thinking about configuring, building, distributing, and installing Erlang systems really ought to spend some time reading Schmidt's thesis. (Or at least the short paper about it by him and Butler Lampson, which is a little easier to find.) Which is where I got "~" from. Pebble and SML (System Modelling Language) come from the same stable. Pebble was mainly used as a formal notation for specifying Cedar, but it _was_ implemented. So "~" has been used in at least two functional languages. And it *does* standardly mean "sort of like =". I grant you that in Haskell it means "match this pattern lazily", and in ML it means "negative sign" as part of a numeric literal, and in C, bizarrely, it means "bitwise complement", but all of those are *unary* uses. See for example http://mathworld.wolfram.com/Tilde.html. The other thing that might make sense would be ":=" (is defined as), although that would tend to confuse people used to sane programming languages that don't use = for assignment. (Fortran has much to answer for. FORmula TRANslator, yet they used .EQ. for equality?) As for =<, that wasn't really a matter of choice. Prolog was used to manipulate logical expressions, and desperately needed <= and => for free use as arrows, forcing equals-or-less than to be written =<. Parlog, FCP, GHC, and Strand 88 all followed it. The first implementation of Erlang was in Prolog. Hence much of Erlang syntax. But not all of it by any means. Erlang *has* innovated. Just not *there*. > > I think I already got used to the proposed frame syntax, followed > your arguments and support it. > > But I think ~ would be great to also roughly mean 'what I just > said'. Also ./. might do, or -"-, or *. It is extremely difficult to believe that this is not an elaborate practical joke. As it happens, I _have_ used a programming language in which * could be used in much that way: Burroughs Extended Algol. From the 1977 manual, page 5-6: Pragmatics An "update assignment" can be specified with an asterisk (*) after the colon equal (:=) by an assignment to an whose is empty. For example, "A := *+1" produces the same results as "A := A+1". Updating a via this method is more efficient." It _should_ have been in the syntax, not the pragmatics. It was more efficient because it only evaluated the subscripts once. I was always rather annoyed because this wasn't consistently carried out: BA[I] := NOT * not allowed RA[I] := * - 1 allowed RA[I] := 1 - * not allowed Note that RA[I] := * * Y + Z is _not_ like C's ra[i] *= y+z, but like ra[i] = (ra[i]*y)+z. That taught me to beware of "what I just said". I mentioned Common Lisp's REPL. SML has something similar: % sml - 6*7; val it = 42 : int - it div 3; val it = 14 : int Now we have two definitions of "it", the later one hiding the earlier. Want to refer to the earlier? You're stuffed. A SINGLE "WHAT-I-JUST-SAID" IS A BAD IDEA NO MATTER HOW YOU SPELL IT. If you are sharp-witted and realise that you need to save a value, you can quickly name it: - val planets = it; val planets = 14 : int - 72; val it = 72 : int - planets; val it = 14 : int In the same way, the S programming language allows assignment to be written right-to-left var <- expr or left-to-right expr -> var so that if you've just typed a hairy long expression and realise that you'll need to refer to the value again, you can tack an assignment on the end without retyping the whole thing. The problem is that we need to *think* about our programs, to take bits of them, manipulate them, put them back, turn them inside out and run them backwards, in a very fluid way. But a "what I just said" is tightly bound to its context. It's not just in a particular place, it's coated with thick layers of superglue, nailed fast, and anchored with a thick chain. Sure, if an expression in an arm of a case refers to a variable bound in the pattern of that arm, it still needs that variable, but it *SAYS* what variable it needs. > The argument about being hard to pronounce sounds a bit convenient > in this context. Pronouncing code is a rarified art and runs into > this problem time again. "Rarefied" (note spelling) means "thin", as in the opposite of "condensed". As for pronouncing code, it SHOULDN'T be an esoteric art. We do, after all, TALK about our programs with each other. What do we do? Point and grunt? I have a tongue and I'm not scared to use it in the service of my profession. "The telephone test" is not of my invention. Here's an example of an error due to the pronunciation of characters: ~1 is the ML way to write what you get by subtracting 1 from 0. If you *pronounce* it "negative one", well and good. Any ML programmer who has got past the tyro stage will get it right. But if you *pronounce* it "minus one", you will far too often write -1 which is a syntax error, if you are lucky. For f x ~1 means (f (x))(~1), while f x -1 means (op -)(f x, 1). The only way I ever found to cure myself of this was to religiously say "negative one". (The same applies in APL, where the "high minus" character is used for "negative".) > But just as '=' can mean many things to different people, maybe '~' > can be used for both things. = has three principal uses in programming languages: query: is this equal to that? definition: this is equal to that, now and always! command: change this to be equal to that (at the moment). "Equal" is in all of them. Maybe ~ *could* be used to mean "what I just said" or "the price of fish in Port Stanley last Thursday" or "this CPU's current clock frequency". Heck, we could even use it to replace self(), which would benefit from not being a function call. That doesn't make any of these a good idea. > I thought it was a tad illogical, however, to argue that because > there is no obviousness to the meaning of <<< in Lisp might indicate > that the 'some as above/before' connotation of ~ should likewise and > somehow guilty by association be irrelevant or even > counterproductive when deciding about. It's use. What I was arguing was perfectly logical: a hack can be turned into a principle if you repeat it *consistently*, but the Lisp * ** *** convention is not repeated *consistently*. (The key one was not the missing <<< but the missing -- and --- .) To put it another way: you *can* demand that people learn some weird new convention, BUT you have to make it pay off for them. Isolated hacks don't pay off for your readers. > Not that I believe that anyone cares, but eye have a hobby horse > that had me thinking about hacking the pre-compiler already: the > doubling of function signatures where the variation that is called > in the former is, plus one default argument as literal, exactly the > head of the next. I might be doing something wrong there but I keep > copying heads and it makes the code hard to read. I regularily go, > copy a head, paste it three times (one over the old ) and alter two > of them marginally. A nice remedy could be: > > Fun(A) -> Fun(A, b). > > Fun(A,B) ... Believe it or not, but this is covered by an old proposal of mine. My adaptation of Paul Lyons' "split procedure heads" covers both keywords and optional arguments. Lisp has (defun f (a (&optional b 'B)) ...) S (R) has f <- function (a, b = "B") { ... } Ada has function F(A : T1; B: T2 => "B") is ... C++ has T0 f(T1 a, T2 b = 'b') { ... } Notice a pattern about where the default value goes? \ > > This could become > > Fun(A) -> ~~ b. That puts the default value where the body belongs, and means that you only get ONE default argument. Two strikes against it to start with. If I wanted this often, I might use M4, and define optional(HEAD, E1, ..., En) to expand to HEAD') -> HEAD'(E1). HEAD' _V_1) -> HEAD' _V_1, E2). ... HEAD' _V_1, ..., V_n_minus_1) -> HEAD' (_V_1, ..., _V_n_minus_1, En). where HEAD' is HEAD without its trailing ), which would be a pretty trivial piece of M4 code, but would let you write optional(foobar(A,B), A+2, 42). foobar(A, B, C, D) -> ... and have it expand to foobar(A,B) -> foobar(A,B, A+2). foobar(A,B, _V_1) -> foobar(A,B, _V_1, 42). foobar(A, B, C, D) -> ... Hmm. I don't quite like that interface. Maybe optional(F(A1, ..., An)) where Ai is either Var or Var=>Expr optional(foobar(A, B=>A+2, C, D=>A*B+C)) That looks like it might work. I did use a special symbol here, but (note again the absence of innovation) it's the same one Ada uses. Come to think of it, since Ada uses => for records as well as optional parameters, it would make sense to use the same symbol in Erlang, and use Var ~ Expr for defaults. Last draft today: optional(foobar(A, B ~ A+2, C, D ~ A*B+C)) In any case, no actual change to Erlang is involved. It would also be possible to hack on the Erlang parser to recognise -optional foobar(A, B ~ A+2, C, D ~ A*B+C). but I'd want to prototype it with M4. > > Sorry there is no relation to the negation connotation that ~ > somehow brings along. But that seemed to be alright for its use in > io:format in the first place, didn't it? Also, somehow, it negates > writing code doubly. If you're a C programmer, sure, ~ says bitwise negation. In Lisp, it doesn't. Not even close. And that's where 'format' comes from. (Possibly via Prolog.) In _Erlang_, the only use of "~" is in formats, and there isn't the slightest whiff of negation about it anywhere. From comptekki@REDACTED Wed May 19 04:59:04 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 20:59:04 -0600 Subject: erl startup In-Reply-To: References: Message-ID: This is odd: If I run: start() -> Id_Table=ets:new(start_id_table, []), {ok, TRef}=timer:apply_interval(1000*60*5, prt, getc, []), % 1000 milliseconds * 60 * 5 = 5 minutes ets:insert(Id_Table, {id, TRef}), io:format("Id_Table: ~w~n", [Id_Table]), io:format("Id_Table lookup: ~w~n", [ets:lookup(Id_Table, id)]), "id_table: " ++ Id_Table. with the -run my_mod the last io:formats print Id_Table: 28693 Id_Table: 28693 Id_Table lookup: [{id,{interval,#Ref<0.0.0.1000>}}] but the "id_table: " ++ Id_Table does not show up in the output. If I then run ets:lookup(28693, id). I get: ** exception error: bad argument in function ets:lookup/2 called as ets:lookup(28693,id) If I run erl then type in my_mod:start() I get : Id_Table: 57386 Id_Table: 57386 Id_Table lookup: [{id,{interval,#Ref<0.0.0.1165>}}] [105,100,95,116,97,98,108,101,58,32|57386] and the timer works now and the last line is output. Any explanation here? Does a timer not continue if it is run from -run command? It seems like it stops working if run in -run mode. thx, -wes From ok@REDACTED Wed May 19 05:01:14 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 19 May 2010 15:01:14 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> Message-ID: <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> On May 18, 2010, at 3:25 PM, Eric Newhuis (personal) wrote: > > And the reconstruction of a small tuple bugged me. Your suggestion > shortens this to something arguably easier to type. > > case X = find(...) of > error -> > {ok, Default}; > {ok, Value} -> > X > end. There are two things here. One is that it gets even simpler: Value can be replaced by a wild-card. The other is that the Haskell idiom for something like this would be to define a function error_default(X = {ok,_}, _Default) -> X; error_default(error, Default) -> {ok,Default}. and then write error_default(find(...), Default) If the Default is an expression which you don't want evaluated at all in the normal case, you might prefer a macro. > > ...without the apparent overhead of reconstruction. (Is it a moot > point due to a clever compiler?) I don't know and I don't care whether the compiler is that smart or not. What I care about is whether a HUMAN reader can see that the value that is returned is the value of find(...). > > But I guess I am still concerned about suggesting this as a general > programming idiom because it introduces a temporary variable. In the name of suffering humanity, WHAT IS SO BAD ABOUT TEMPORARY VARIABLES? I grant you that there were software engineering proscriptions of "temporary variables" in the sense of scratch variables that got reused over and over with no specific semantics of their own. But the variables we are talking about here are no more "temporary" than any other Erlang variables. If you mean that the variable is used over a very narrow range, that's the direct *opposite* of what the preaching against temporary variables was about, and there is no harm in it at all, as long as you choose your names well. > And as such one would need to study pre- and post-logic to avoid > accidental use of the same variable name to name something different. I'm sorry, but (A) this is trivial. Tell your editor "mark clause as region". [A simple hack for this is to go back to the first line that starts with a letter, apostrophe, or dash, then forward to just before the next such line. I don't actually have this command at the moment, so I'd do Ctrl-_ Meta-Ctrl-R, Ctrl-@, Meta-Ctrl-R.] Then tell your editor "list variables in region". [I don't have this at the moment, but I do have code for Prolog that I could adapt to do it.] The Smalltalk IDE I often use already has "list variables in scope". (B) Even without that, it's trivial. Mark the region as before, then "find word within region". Don't find it? You're safe. (C) I am at a loss to think of what kind of programming would make this a problem. You are making up a name to stand for a meaning that doesn't _have_ a name yet. Of *course* it gets a name that is different from the other names; it *means* something different so you give it a different name automatically. X is just an example that's been used in a vacuum. In real life, with real examples, in reasonable Erlang code, there shouldn't be a problem, especially if you use little functions like the error_default/2 example above (which can be inlined by the compiler). > > I'd not have this concern if Erlang had a LET construct: True. You would have a different concern, that of accidentally hiding a variable you needed to refer to. It wouldn't be substantially different. > So apparently my idea isn't so great. And I also don't have a > satisfying temporary-free and reconstruction-free solution. I don't have any solution to the problem of programming Erlang without using the letter 'e' either. I'm not worried about that. I don't see any problem with introducing variable names as long as they are meaningful in context. That's not Haskell style, but Erlang isn't Haskell. I don't see any problem with moving stuff out into other functions that are then inlined, above all, if the schema is one you can use again, it's a _good_ thing to do that. "Temporary variables" that are in another function entirely aren't any problem, are they? I think this is one huge issue: DON'T BE SCARED OF SMALL FUNCTIONS. Oh, and let's have real examples, please! > From mscame@REDACTED Wed May 19 05:10:29 2010 From: mscame@REDACTED (William Yangzhuwei) Date: Wed, 19 May 2010 11:10:29 +0800 Subject: what is sysI module ? Message-ID: hi there, I am a erlang beginner. Does anyone know what is sysI module? like sysI:spawn_opts(server, [] ). ? I cannot find any doc from Internet regarding this. Anyone can help to figure out? thanks! -- Best Regards //William YangZhuwei From enewhuis@REDACTED Wed May 19 05:11:13 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Tue, 18 May 2010 22:11:13 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> Message-ID: I appreciate all your comments. I still don't like temporary variables like X or whatever because they contain no semantic quality and there is a tension between short names to make the pattern obvious and longer names for reunderstanding. (Oh, I write plenty of small functions. Its small functions all the way down.) But I should stop interrupting you with something so trivial so you can get those frames implemented. I loathe the record syntax. On May 18, 2010, at 10:01 PM, Richard O'Keefe wrote: > > On May 18, 2010, at 3:25 PM, Eric Newhuis (personal) wrote: >> >> And the reconstruction of a small tuple bugged me. Your suggestion shortens this to something arguably easier to type. >> >> case X = find(...) of >> error -> >> {ok, Default}; >> {ok, Value} -> >> X >> end. > > There are two things here. One is that it gets even simpler: > Value can be replaced by a wild-card. The other is that the > Haskell idiom for something like this would be to define a > function > > error_default(X = {ok,_}, _Default) -> X; > error_default(error, Default) -> {ok,Default}. > > and then write > > error_default(find(...), Default) > > If the Default is an expression which you don't want evaluated at > all in the normal case, you might prefer a macro. >> >> ...without the apparent overhead of reconstruction. (Is it a moot point due to a clever compiler?) > > I don't know and I don't care whether the compiler is that smart or > not. What I care about is whether a HUMAN reader can see that the > value that is returned is the value of find(...). >> >> But I guess I am still concerned about suggesting this as a general programming idiom because it introduces a temporary variable. > > In the name of suffering humanity, > WHAT IS SO BAD ABOUT TEMPORARY VARIABLES? > > I grant you that there were software engineering proscriptions of > "temporary variables" in the sense of scratch variables that got > reused over and over with no specific semantics of their own. > But the variables we are talking about here are no more "temporary" > than any other Erlang variables. If you mean that the variable is > used over a very narrow range, that's the direct *opposite* of > what the preaching against temporary variables was about, and there > is no harm in it at all, as long as you choose your names well. > >> And as such one would need to study pre- and post-logic to avoid accidental use of the same variable name to name something different. > > I'm sorry, but > (A) this is trivial. Tell your editor "mark clause as region". > [A simple hack for this is to go back to the first line that > starts with a letter, apostrophe, or dash, then forward to just > before the next such line. I don't actually have this command > at the moment, so I'd do Ctrl-_ Meta-Ctrl-R, Ctrl-@, Meta-Ctrl-R.] > Then tell your editor "list variables in region". [I don't have > this at the moment, but I do have code for Prolog that I could > adapt to do it.] The Smalltalk IDE I often use already has > "list variables in scope". > > (B) Even without that, it's trivial. Mark the region as before, > then "find word within region". Don't find it? You're safe. > > (C) I am at a loss to think of what kind of programming would make > this a problem. You are making up a name to stand for a meaning > that doesn't _have_ a name yet. Of *course* it gets a name that > is different from the other names; it *means* something different > so you give it a different name automatically. > > X is just an example that's been used in a vacuum. > In real life, with real examples, in reasonable Erlang code, > there shouldn't be a problem, especially if you use little > functions like the error_default/2 example above (which can be > inlined by the compiler). >> >> I'd not have this concern if Erlang had a LET construct: > > True. You would have a different concern, that of accidentally > hiding a variable you needed to refer to. It wouldn't be > substantially different. > >> So apparently my idea isn't so great. And I also don't have a satisfying temporary-free and reconstruction-free solution. > > I don't have any solution to the problem of programming Erlang > without using the letter 'e' either. I'm not worried about that. > > I don't see any problem with introducing variable names > as long as they are meaningful in context. That's not Haskell > style, but Erlang isn't Haskell. > > I don't see any problem with moving stuff out into other functions > that are then inlined, above all, if the schema is one you can use > again, it's a _good_ thing to do that. "Temporary variables" that > are in another function entirely aren't any problem, are they? > > I think this is one huge issue: > > DON'T BE SCARED OF SMALL FUNCTIONS. > > Oh, and let's have real examples, please! > >> From bernie@REDACTED Wed May 19 05:11:27 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 19 May 2010 13:11:27 +1000 Subject: [erlang-questions] Re: erl startup In-Reply-To: References: Message-ID: <4BF356DF.7040804@m5net.com> On 19/05/10 12:59, Wes James wrote: > Any explanation here? > Yep. My guess is that -run either spawns your function in a new process or (more likely) runs it in a process which subsequently exits. That means that: a) The timer will stop (since the process that launched it shut down). b) The return value ("id_table: " ++ Id_Table) will not be printed (since there's no command to print it). c) The ets table will no longer exist (since, again, the process that spawned it has finished and you did not specify an heir), hence the badarg when you try to reference it. By contrast, when you run my_mod:start() from the shell, the shell process that launched it continues to run (so the timer and the ets table persist), and the return value is printed (as happens with any other function you call from the shell). B From comptekki@REDACTED Wed May 19 05:22:37 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 21:22:37 -0600 Subject: [erlang-questions] Re: erl startup In-Reply-To: <4BF356DF.7040804@m5net.com> References: <4BF356DF.7040804@m5net.com> Message-ID: On Tue, May 18, 2010 at 9:11 PM, Bernard Duggan wrote: > On 19/05/10 12:59, Wes James wrote: >> >> Any explanation here? >> > > Yep. > > My guess is that -run either spawns your function in a new process or (more > likely) runs it in a process which subsequently exits. ?That means that: > > a) The timer will stop (since the process that launched it shut down). > b) The return value ("id_table: " ++ Id_Table) will not be printed (since > there's no command to print it). > c) The ets table will no longer exist (since, again, the process that > spawned it has finished and you did not specify an heir), hence the badarg > when you try to reference it. > > By contrast, when you run my_mod:start() from the shell, the shell process > that launched it continues to run (so the timer and the ets table persist), > and the return value is printed (as happens with any other function you call > from the shell). > > B > Bernard, Thx. I guess I need to look at spawning this?? -wes From ok@REDACTED Wed May 19 05:30:33 2010 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 19 May 2010 15:30:33 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> Message-ID: On May 19, 2010, at 3:11 PM, Eric Newhuis (personal) wrote: > I appreciate all your comments. > > I still don't like temporary variables like X or whatever because > they contain no semantic quality and there is a tension between > short names to make the pattern obvious and longer names for > reunderstanding. The variable name X has been used in the examples, because the examples have been free of content. No-one, as far as I am aware, is arguing for the use of X _as such_, although it is at least better than _ or ~ . This is why I keep on screaming for *real* examples. I predict that in real examples, we can find a way to express what we want with meaningful variable names. From bernie@REDACTED Wed May 19 05:35:02 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 19 May 2010 13:35:02 +1000 Subject: [erlang-questions] Re: erl startup In-Reply-To: References: <4BF356DF.7040804@m5net.com> Message-ID: <4BF35C66.2040505@m5net.com> On 19/05/10 13:22, Wes James wrote: > Thx. I guess I need to look at spawning this?? > Sure, if you like, but it won't change any of the behaviour you're seeing. You'll spawn a process to run the function, the function will run, the function will finsih and so the process will exit and you'll be exactly where you are now. In your first posts, you were comparing the behaviour of your single function with the /applications/ yaws and mnesia. A single function does not an application make. Perhaps if you explained what exactly you're trying to achieve it would be easier to provide specific guidance. B From hd2010@REDACTED Wed May 19 05:36:54 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 19 May 2010 05:36:54 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> Message-ID: <4BF35CD6.6010708@eonblast.com> But for those who crave, does anybody have a tip where to look in the precompiler sources to add a _ or ~ or * the way Eric asked for? Just a starting point to mod would be cool. Thanks, Henning Richard O'Keefe wrote: > > On May 19, 2010, at 3:11 PM, Eric Newhuis (personal) wrote: > >> I appreciate all your comments. >> >> I still don't like temporary variables like X or whatever because >> they contain no semantic quality and there is a tension between short >> names to make the pattern obvious and longer names for reunderstanding. > > The variable name X has been used in the examples, > because the examples have been free of content. > No-one, as far as I am aware, is arguing for the use of X > _as such_, although it is at least better than _ or ~ . > > This is why I keep on screaming for *real* examples. > I predict that in real examples, we can find a way to express > what we want with meaningful variable names. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Wed May 19 05:46:39 2010 From: comptekki@REDACTED (Wes James) Date: Tue, 18 May 2010 21:46:39 -0600 Subject: [erlang-questions] Re: erl startup In-Reply-To: <4BF35C66.2040505@m5net.com> References: <4BF356DF.7040804@m5net.com> <4BF35C66.2040505@m5net.com> Message-ID: On Tue, May 18, 2010 at 9:35 PM, Bernard Duggan wrote: > On 19/05/10 13:22, Wes James wrote: >> >> Thx. ?I guess I need to look at spawning this?? >> > > ? ?Sure, if you like, but it won't change any of the behaviour you're > seeing. ?You'll spawn a process to run the function, the function will run, > the function will finsih and so the process will exit and you'll be exactly > where you are now. > ? ?In your first posts, you were comparing the behaviour of your single > function with the /applications/ yaws and mnesia. ?A single function does > not an application make. ?Perhaps if you explained what exactly you're > trying to achieve it would be easier to provide specific guidance. > > B > I'm trying run my_mod which i'm running in conjunction with yaws so that the my_mod will every 5 minutes go and check a printer for home and grab the pages printed and stick it in a mnesia table. I then use a .yaws page to look at the mnesia page count values. This is why I'm trying to get the timer to work as I start yaws and keep going after yaws is started. thx, -wes From hd2010@REDACTED Wed May 19 06:04:12 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 19 May 2010 06:04:12 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <760424A5-1E77-4A3B-85EE-0168B23FA48F@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <4BF1D8DD.8080000@eonblast.com> <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> <4BF203E1.9070905@eonblast.com> <760424A5-1E77-4A3B-85EE-0168B23FA48F@cs.otago.ac.nz> Message-ID: <4BF3633C.3070406@eonblast.com> True pleasure as always Sir, so it's a done deal and we'll make a proposal for foobar(A, B ~ A+2, C, D ~ A*B+C) -> ... ? Or are you insisting on M4 for this? For the record, the negation connotation was raised by you earlier. > I am baffled by this "so completely special" claim. With that disclaimer, "to my uneducated eyes" - yes, certainly. I had no exposure to Prolog. It's premise had been suspect to me since high school and I successfully avoided it. I like the idea of being consistent about hacks to make them justifiable. But for now, why not find characters, or even reserveable atoms, for these desired uses. Let's not forget `_ -> ok.' either. I will look out for samples in my sources where I would benefit from: foobar(A, B ~ A+2, C, D ~ A*B+C) -> ... := foobar(A, D) -> foobar(A, A+2, C,A*B+C). foobar(A, B, D) -> foobar(A, B, C,A*B+C). foobar(A, B, C, D) -> ... - - - - - - - - - - case foo() of {ping,_} -> ~~ ; {_,pong} -> pang(~~) end := case X=foo() of {ping,_} -> X ; {_,pong} -> pang(X) end - - - - - - - - - - case foo() of { something, And } -> bar(And); ~~~ end := case X = foo() of { something, And } -> bar(And); _ -> X end Or any other sign. Although sorta-kinda makes sense to me. The third even looks nicely formatted. Not that I couldn't live without. But I dare participating in the conversation for this reason: after a while you are getting used to the funniest things. When you're fresh it's easier to note what could be different, while at the same time unable to judge the deeper consequences. > But a "what I just said" is tightly bound to its context. > It's not just in a particular place, it's coated with thick > layers of superglue, nailed fast, and anchored with a thick > chain. Sure, if an expression in an arm of a case refers to > a variable bound in the pattern of that arm, it still needs > that variable, but it *SAYS* what variable it needs. I don't think that makes a difference, you still need to the context to make sense of it. Named or not. > A SINGLE "WHAT-I-JUST-SAID" IS A BAD IDEA NO MATTER HOW YOU SPELL IT. For my taste, constructions can make sense even if you can't really stretch their use. I even found macro implemented templates somewhat better than none, or compiler constructs with their usual limitations (if you don't happen to have you M4 at hand, of course). > "Rarefied" (note spelling) We can absolutely switch to German if you feel its essential that I insert no human language related mistakes. > As for pronouncing code, it SHOULDN'T be an esoteric art. Maybe, but it most certainly is. I would not enjoy reading an source over the phone and I also don't think that's the point of PLs. I mean, Erlang??? Open curly brace, Open square brace, Open curly brace, one, close curly brace ... wow. I maintain that a tilde by convention named 'sorta-kina-operator' would be less of a nuisance to pronounce than all the brackets the syntax of Erlang loves so much. And fun. > =< and >= I didn't doubt there was a reason but thanks for the education. I was pretty sure it was to reserve <= and => just wondered that there weren't any of these before binary matching. Thanks! Henning Richard O'Keefe wrote: > > On May 18, 2010, at 3:05 PM, Henning Diedrich wrote: > >> Richard, righteous, >> >> I was just curious about the defense you'd mount to keep ~ for the >> frame syntax. >> >> Where the first thing that hit my uneducated I was how the use of ~ >> in it was so completely special --- as some other symbol uses in >> Erlang are --- that it, well, may be an excellent fit. Why starting >> to get conventional. A language proudly flying =< could also have ~ >> to mean 'something like' while really, yes, significantly different >> from =. > > I am baffled by this "so completely special" claim. > The frames report goes to some trouble to point out that this use of > ~ for maplets is *NOT* innovative. It was a straight steal from > Xerox PARC. The Mesa programming language had an equivalent of > 'make' called C/Mesa. For Mesa's successor Cedar, this was replaced > by Eric Schimdt's "System Modeller", which took care of > - versions > - building > - distribution > using a "system model" (think "Makefile on steroids") which was nothing > other than a pure lazy functional program for building systems. (This > is the same Eric Schmidt who became CEO of Google.) A lovely idea, > and anyone thinking about configuring, building, distributing, and > installing Erlang systems really ought to spend some time reading > Schmidt's thesis. (Or at least the short paper about it by him and > Butler Lampson, which is a little easier to find.) > > Which is where I got "~" from. Pebble and SML (System Modelling > Language) come from the same stable. Pebble was mainly used as a > formal notation for specifying Cedar, but it _was_ implemented. > So "~" has been used in at least two functional languages. > > And it *does* standardly mean "sort of like =". I grant you that in > Haskell it means "match this pattern lazily", and in ML it means > "negative sign" as part of a numeric literal, and in C, bizarrely, > it means "bitwise complement", but all of those are *unary* uses. > See for example http://mathworld.wolfram.com/Tilde.html. > > The other thing that might make sense would be ":=" (is defined as), > although that would tend to confuse people used to sane programming > languages that don't use = for assignment. (Fortran has much to > answer for. FORmula TRANslator, yet they used .EQ. for equality?) > > As for =<, that wasn't really a matter of choice. Prolog was used > to manipulate logical expressions, and desperately needed <= and => > for free use as arrows, forcing equals-or-less than to be written > =<. Parlog, FCP, GHC, and Strand 88 all followed it. The first > implementation of Erlang was in Prolog. Hence much of Erlang > syntax. But not all of it by any means. Erlang *has* innovated. > Just not *there*. > > >> >> I think I already got used to the proposed frame syntax, followed >> your arguments and support it. >> >> But I think ~ would be great to also roughly mean 'what I just said'. >> Also ./. might do, or -"-, or *. > > It is extremely difficult to believe that this is not an elaborate > practical joke. > > As it happens, I _have_ used a programming language in which * could be > used in much that way: Burroughs Extended Algol. From the 1977 manual, > page 5-6: > Pragmatics > An "update assignment" can be specified with an asterisk (*) > after the colon equal (:=) by an assignment to an > whose is empty. > For example, "A := *+1" produces the same results as > "A := A+1". Updating a via this > method is more efficient." > > It _should_ have been in the syntax, not the pragmatics. > It was more efficient because it only evaluated the subscripts once. > > I was always rather annoyed because this wasn't consistently carried > out: > BA[I] := NOT * not allowed > RA[I] := * - 1 allowed > RA[I] := 1 - * not allowed > Note that > RA[I] := * * Y + Z > is _not_ like C's ra[i] *= y+z, but like ra[i] = (ra[i]*y)+z. > > That taught me to beware of "what I just said". > > I mentioned Common Lisp's REPL. SML has something similar: > % sml > - 6*7; > val it = 42 : int > - it div 3; > val it = 14 : int > Now we have two definitions of "it", the later one hiding the > earlier. Want to refer to the earlier? You're stuffed. > > A SINGLE "WHAT-I-JUST-SAID" IS A BAD IDEA NO MATTER HOW YOU SPELL IT. > > If you are sharp-witted and realise that you need to save a value, > you can quickly name it: > > - val planets = it; > val planets = 14 : int > - 72; > val it = 72 : int > - planets; > val it = 14 : int > > In the same way, the S programming language allows assignment to > be written right-to-left > var <- expr > or left-to-right > expr -> var > so that if you've just typed a hairy long expression and realise > that you'll need to refer to the value again, you can tack an > assignment on the end without retyping the whole thing. > > The problem is that we need to *think* about our programs, to take > bits of them, manipulate them, put them back, turn them inside out > and run them backwards, in a very fluid way. > > But a "what I just said" is tightly bound to its context. > It's not just in a particular place, it's coated with thick > layers of superglue, nailed fast, and anchored with a thick > chain. Sure, if an expression in an arm of a case refers to > a variable bound in the pattern of that arm, it still needs > that variable, but it *SAYS* what variable it needs. > >> The argument about being hard to pronounce sounds a bit convenient in >> this context. Pronouncing code is a rarified art and runs into this >> problem time again. > > "Rarefied" (note spelling) means "thin", as in the opposite of > "condensed". As for pronouncing code, it SHOULDN'T be an esoteric > art. We do, after all, TALK about our programs with each other. > What do we do? Point and grunt? I have a tongue and I'm not scared > to use it in the service of my profession. "The telephone test" is > not of my invention. > > Here's an example of an error due to the pronunciation of > characters: > ~1 > is the ML way to write what you get by subtracting 1 from 0. > If you *pronounce* it "negative one", well and good. Any ML > programmer who has got past the tyro stage will get it right. > But if you *pronounce* it "minus one", you will far too often > write > -1 > which is a syntax error, if you are lucky. For > f x ~1 > means (f (x))(~1), while > f x -1 > means (op -)(f x, 1). The only way I ever found to cure myself > of this was to religiously say "negative one". (The same applies > in APL, where the "high minus" character is used for "negative".) > >> But just as '=' can mean many things to different people, maybe '~' >> can be used for both things. > > = has three principal uses in programming languages: > query: is this equal to that? > definition: this is equal to that, now and always! > command: change this to be equal to that (at the moment). > "Equal" is in all of them. > > Maybe ~ *could* be used to mean "what I just said" or "the price > of fish in Port Stanley last Thursday" or "this CPU's current clock > frequency". Heck, we could even use it to replace self(), which > would benefit from not being a function call. That doesn't make > any of these a good idea. > >> I thought it was a tad illogical, however, to argue that because >> there is no obviousness to the meaning of <<< in Lisp might indicate >> that the 'some as above/before' connotation of ~ should likewise and >> somehow guilty by association be irrelevant or even counterproductive >> when deciding about. It's use. > > What I was arguing was perfectly logical: a hack can be turned into a > principle if you repeat it *consistently*, but the Lisp * ** *** > convention is not repeated *consistently*. (The key one was not the > missing <<< but the missing -- and --- .) > > To put it another way: you *can* demand that people learn some weird > new convention, BUT you have to make it pay off for them. Isolated > hacks don't pay off for your readers. > >> Not that I believe that anyone cares, but eye have a hobby horse that >> had me thinking about hacking the pre-compiler already: the doubling >> of function signatures where the variation that is called in the >> former is, plus one default argument as literal, exactly the head of >> the next. I might be doing something wrong there but I keep copying >> heads and it makes the code hard to read. I regularily go, copy a >> head, paste it three times (one over the old ) and alter two of them >> marginally. A nice remedy could be: >> >> Fun(A) -> Fun(A, b). >> >> Fun(A,B) ... > > Believe it or not, but this is covered by an old proposal of mine. > My adaptation of Paul Lyons' "split procedure heads" covers both > keywords and optional arguments. > > Lisp has (defun f (a (&optional b 'B)) ...) > S (R) has f <- function (a, b = "B") { ... } > Ada has function F(A : T1; B: T2 => "B") is ... > C++ has T0 f(T1 a, T2 b = 'b') { ... } > > Notice a pattern about where the default value goes? > \ >> >> This could become >> >> Fun(A) -> ~~ b. > > That puts the default value where the body belongs, > and means that you only get ONE default argument. > Two strikes against it to start with. > > If I wanted this often, I might use M4, and define > > optional(HEAD, E1, ..., En) > > to expand to > > HEAD') -> HEAD'(E1). > HEAD' _V_1) -> HEAD' _V_1, E2). > ... > HEAD' _V_1, ..., V_n_minus_1) -> HEAD' (_V_1, ..., _V_n_minus_1, En). > > where HEAD' is HEAD without its trailing ), > which would be a pretty trivial piece of M4 code, but would > let you write > > optional(foobar(A,B), A+2, 42). > foobar(A, B, C, D) -> ... > > and have it expand to > > foobar(A,B) -> foobar(A,B, A+2). > foobar(A,B, _V_1) -> foobar(A,B, _V_1, 42). > foobar(A, B, C, D) -> ... > > Hmm. I don't quite like that interface. Maybe > > optional(F(A1, ..., An)) > > where Ai is either Var or Var=>Expr > > optional(foobar(A, B=>A+2, C, D=>A*B+C)) > > That looks like it might work. I did use a special symbol here, > but (note again the absence of innovation) it's the same one Ada > uses. Come to think of it, since Ada uses => for records as well > as optional parameters, it would make sense to use the same > symbol in Erlang, and use Var ~ Expr for defaults. > > Last draft today: > > optional(foobar(A, B ~ A+2, C, D ~ A*B+C)) > > In any case, no actual change to Erlang is involved. > It would also be possible to hack on the Erlang parser > to recognise > -optional foobar(A, B ~ A+2, C, D ~ A*B+C). > but I'd want to prototype it with M4. > >> >> Sorry there is no relation to the negation connotation that ~ somehow >> brings along. But that seemed to be alright for its use in io:format >> in the first place, didn't it? Also, somehow, it negates writing code >> doubly. > > If you're a C programmer, sure, ~ says bitwise negation. > In Lisp, it doesn't. Not even close. And that's where 'format' > comes from. (Possibly via Prolog.) In _Erlang_, the only use > of "~" is in formats, and there isn't the slightest whiff of > negation about it anywhere. > > > From bernie@REDACTED Wed May 19 06:14:02 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 19 May 2010 14:14:02 +1000 Subject: [erlang-questions] Re: erl startup In-Reply-To: References: <4BF356DF.7040804@m5net.com> <4BF35C66.2040505@m5net.com> Message-ID: <4BF3658A.7070208@m5net.com> On 19/05/10 13:46, Wes James wrote: > > I'm trying run my_mod which i'm running in conjunction with yaws so > that the my_mod will every 5 minutes go and check a printer for home > and grab the pages printed and stick it in a mnesia table. I then use > a .yaws page to look at the mnesia page count values. This is why I'm > trying to get the timer to work as I start yaws and keep going after > yaws is started. > Okay, so it seems like this issue here is the notion of "running a module". I don't think it means what you think it means (actually I don't think it means anything at all, really). What it sounds like you want to do is spawn a process that does some startup stuff and then basically sits idle to allow its ets table and timer to continue to work. There's a couple of ways to approach this - the "OTP way(s)" would be to either wrap the whole thing in an application or gen_server behaviour. That gives you all sorts of nice extras (easy supervision, appearing in appmon etc) but does require a bit of scaffold code and has something of a learning curve before you figure out how to do it "right" (hell, I'm /still/ figuring the details of that out). The much easier (though less flexible and powerful) way is to modify your code to something like this: start() -> spawn_link(?MODULE, run, []). run() -> Id_Table=ets:new(start_id_table, []), io:format("~w~n", [Id_Table]), {ok, TRef}=timer:apply_interval(1000*60*5, prt, getc, []), % 1000 milliseconds * 60 * 5 = 5 minutes ets:insert(Id_Table, {id, TRef}), wait_for_exit(). wait_for_exit() -> receive some_exit_signal -> ok; _ -> wait_for_exit() end. (Haven't compile or tested this - there's probably a syntax error or two). Note the tail-recursive receive loop at the end which keeps the process alive and keeps its message queue empty but does nothing else. Also, just as a tip: I recently discovered the timer:minutes/1 function and its friends - much easier to read and code than "1000*60*5" :) Cheers, Bernard From bengt.kleberg@REDACTED Wed May 19 07:08:49 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 19 May 2010 07:08:49 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <760424A5-1E77-4A3B-85EE-0168B23FA48F@cs.otago.ac.nz> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <4BF1D8DD.8080000@eonblast.com> <9ED1C2B4-01D9-41F4-8411-90306A448ACA@cs.otago.ac.nz> <4BF203E1.9070905@eonblast.com> <760424A5-1E77-4A3B-85EE-0168B23FA48F@cs.otago.ac.nz> Message-ID: <1274245730.4976.3.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> I think this is the short paper by Schmidt and Lampson: http://research.microsoft.com/en-us/um/people/blampson/32-organizingsoftware/32-organizingsoftwareocr.doc Unfortunately it is unavailable there, but there is a cache: http://webcache.googleusercontent.com/search?q=cache:YMjF4RmG6BEJ:http://research.microsoft.com/en-us/um/people/blampson/32-organizingsoftware/32-organizingsoftwareocr.doc+cedar+Eric+Schmidt+system+model&hl=&ct=clnk bengt On Wed, 2010-05-19 at 04:41 +0200, Richard O'Keefe wrote: > On May 18, 2010, at 3:05 PM, Henning Diedrich wrote: > > > Richard, righteous, > > > > I was just curious about the defense you'd mount to keep ~ for the > > frame syntax. > > > > Where the first thing that hit my uneducated I was how the use of ~ > > in it was so completely special --- as some other symbol uses in > > Erlang are --- that it, well, may be an excellent fit. Why starting > > to get conventional. A language proudly flying =< could also have ~ > > to mean 'something like' while really, yes, significantly different > > from =. > > I am baffled by this "so completely special" claim. > The frames report goes to some trouble to point out that this use of > ~ for maplets is *NOT* innovative. It was a straight steal from > Xerox PARC. The Mesa programming language had an equivalent of > 'make' called C/Mesa. For Mesa's successor Cedar, this was replaced > by Eric Schimdt's "System Modeller", which took care of > - versions > - building > - distribution > using a "system model" (think "Makefile on steroids") which was nothing > other than a pure lazy functional program for building systems. (This > is the same Eric Schmidt who became CEO of Google.) A lovely idea, > and anyone thinking about configuring, building, distributing, and > installing Erlang systems really ought to spend some time reading > Schmidt's thesis. (Or at least the short paper about it by him and > Butler Lampson, which is a little easier to find.) > > Which is where I got "~" from. Pebble and SML (System Modelling > Language) come from the same stable. Pebble was mainly used as a > formal notation for specifying Cedar, but it _was_ implemented. > So "~" has been used in at least two functional languages. > > And it *does* standardly mean "sort of like =". I grant you that in > Haskell it means "match this pattern lazily", and in ML it means > "negative sign" as part of a numeric literal, and in C, bizarrely, > it means "bitwise complement", but all of those are *unary* uses. > See for example http://mathworld.wolfram.com/Tilde.html. > > The other thing that might make sense would be ":=" (is defined as), > although that would tend to confuse people used to sane programming > languages that don't use = for assignment. (Fortran has much to > answer for. FORmula TRANslator, yet they used .EQ. for equality?) > > As for =<, that wasn't really a matter of choice. Prolog was used > to manipulate logical expressions, and desperately needed <= and => > for free use as arrows, forcing equals-or-less than to be written > =<. Parlog, FCP, GHC, and Strand 88 all followed it. The first > implementation of Erlang was in Prolog. Hence much of Erlang > syntax. But not all of it by any means. Erlang *has* innovated. > Just not *there*. > > > > > > I think I already got used to the proposed frame syntax, followed > > your arguments and support it. > > > > But I think ~ would be great to also roughly mean 'what I just > > said'. Also ./. might do, or -"-, or *. > > It is extremely difficult to believe that this is not an elaborate > practical joke. > > As it happens, I _have_ used a programming language in which * could be > used in much that way: Burroughs Extended Algol. From the 1977 manual, > page 5-6: > Pragmatics > An "update assignment" can be specified with an asterisk (*) > after the colon equal (:=) by an assignment to an > whose is empty. > For example, "A := *+1" produces the same results as > "A := A+1". Updating a via this > method is more efficient." > > It _should_ have been in the syntax, not the pragmatics. > It was more efficient because it only evaluated the subscripts once. > > I was always rather annoyed because this wasn't consistently carried > out: > BA[I] := NOT * not allowed > RA[I] := * - 1 allowed > RA[I] := 1 - * not allowed > Note that > RA[I] := * * Y + Z > is _not_ like C's ra[i] *= y+z, but like ra[i] = (ra[i]*y)+z. > > That taught me to beware of "what I just said". > > I mentioned Common Lisp's REPL. SML has something similar: > % sml > - 6*7; > val it = 42 : int > - it div 3; > val it = 14 : int > Now we have two definitions of "it", the later one hiding the > earlier. Want to refer to the earlier? You're stuffed. > > A SINGLE "WHAT-I-JUST-SAID" IS A BAD IDEA NO MATTER HOW YOU SPELL IT. > > If you are sharp-witted and realise that you need to save a value, > you can quickly name it: > > - val planets = it; > val planets = 14 : int > - 72; > val it = 72 : int > - planets; > val it = 14 : int > > In the same way, the S programming language allows assignment to > be written right-to-left > var <- expr > or left-to-right > expr -> var > so that if you've just typed a hairy long expression and realise > that you'll need to refer to the value again, you can tack an > assignment on the end without retyping the whole thing. > > The problem is that we need to *think* about our programs, to take > bits of them, manipulate them, put them back, turn them inside out > and run them backwards, in a very fluid way. > > But a "what I just said" is tightly bound to its context. > It's not just in a particular place, it's coated with thick > layers of superglue, nailed fast, and anchored with a thick > chain. Sure, if an expression in an arm of a case refers to > a variable bound in the pattern of that arm, it still needs > that variable, but it *SAYS* what variable it needs. > > > The argument about being hard to pronounce sounds a bit convenient > > in this context. Pronouncing code is a rarified art and runs into > > this problem time again. > > "Rarefied" (note spelling) means "thin", as in the opposite of > "condensed". As for pronouncing code, it SHOULDN'T be an esoteric > art. We do, after all, TALK about our programs with each other. > What do we do? Point and grunt? I have a tongue and I'm not scared > to use it in the service of my profession. "The telephone test" is > not of my invention. > > Here's an example of an error due to the pronunciation of > characters: > ~1 > is the ML way to write what you get by subtracting 1 from 0. > If you *pronounce* it "negative one", well and good. Any ML > programmer who has got past the tyro stage will get it right. > But if you *pronounce* it "minus one", you will far too often > write > -1 > which is a syntax error, if you are lucky. For > f x ~1 > means (f (x))(~1), while > f x -1 > means (op -)(f x, 1). The only way I ever found to cure myself > of this was to religiously say "negative one". (The same applies > in APL, where the "high minus" character is used for "negative".) > > > But just as '=' can mean many things to different people, maybe '~' > > can be used for both things. > > = has three principal uses in programming languages: > query: is this equal to that? > definition: this is equal to that, now and always! > command: change this to be equal to that (at the moment). > "Equal" is in all of them. > > Maybe ~ *could* be used to mean "what I just said" or "the price > of fish in Port Stanley last Thursday" or "this CPU's current clock > frequency". Heck, we could even use it to replace self(), which > would benefit from not being a function call. That doesn't make > any of these a good idea. > > > I thought it was a tad illogical, however, to argue that because > > there is no obviousness to the meaning of <<< in Lisp might indicate > > that the 'some as above/before' connotation of ~ should likewise and > > somehow guilty by association be irrelevant or even > > counterproductive when deciding about. It's use. > > What I was arguing was perfectly logical: a hack can be turned into a > principle if you repeat it *consistently*, but the Lisp * ** *** > convention is not repeated *consistently*. (The key one was not the > missing <<< but the missing -- and --- .) > > To put it another way: you *can* demand that people learn some weird > new convention, BUT you have to make it pay off for them. Isolated > hacks don't pay off for your readers. > > > Not that I believe that anyone cares, but eye have a hobby horse > > that had me thinking about hacking the pre-compiler already: the > > doubling of function signatures where the variation that is called > > in the former is, plus one default argument as literal, exactly the > > head of the next. I might be doing something wrong there but I keep > > copying heads and it makes the code hard to read. I regularily go, > > copy a head, paste it three times (one over the old ) and alter two > > of them marginally. A nice remedy could be: > > > > Fun(A) -> Fun(A, b). > > > > Fun(A,B) ... > > Believe it or not, but this is covered by an old proposal of mine. > My adaptation of Paul Lyons' "split procedure heads" covers both > keywords and optional arguments. > > Lisp has (defun f (a (&optional b 'B)) ...) > S (R) has f <- function (a, b = "B") { ... } > Ada has function F(A : T1; B: T2 => "B") is ... > C++ has T0 f(T1 a, T2 b = 'b') { ... } > > Notice a pattern about where the default value goes? > \ > > > > This could become > > > > Fun(A) -> ~~ b. > > That puts the default value where the body belongs, > and means that you only get ONE default argument. > Two strikes against it to start with. > > If I wanted this often, I might use M4, and define > > optional(HEAD, E1, ..., En) > > to expand to > > HEAD') -> HEAD'(E1). > HEAD' _V_1) -> HEAD' _V_1, E2). > ... > HEAD' _V_1, ..., V_n_minus_1) -> HEAD' (_V_1, ..., _V_n_minus_1, En). > > where HEAD' is HEAD without its trailing ), > which would be a pretty trivial piece of M4 code, but would > let you write > > optional(foobar(A,B), A+2, 42). > foobar(A, B, C, D) -> ... > > and have it expand to > > foobar(A,B) -> foobar(A,B, A+2). > foobar(A,B, _V_1) -> foobar(A,B, _V_1, 42). > foobar(A, B, C, D) -> ... > > Hmm. I don't quite like that interface. Maybe > > optional(F(A1, ..., An)) > > where Ai is either Var or Var=>Expr > > optional(foobar(A, B=>A+2, C, D=>A*B+C)) > > That looks like it might work. I did use a special symbol here, > but (note again the absence of innovation) it's the same one Ada > uses. Come to think of it, since Ada uses => for records as well > as optional parameters, it would make sense to use the same > symbol in Erlang, and use Var ~ Expr for defaults. > > Last draft today: > > optional(foobar(A, B ~ A+2, C, D ~ A*B+C)) > > In any case, no actual change to Erlang is involved. > It would also be possible to hack on the Erlang parser > to recognise > -optional foobar(A, B ~ A+2, C, D ~ A*B+C). > but I'd want to prototype it with M4. > > > > > Sorry there is no relation to the negation connotation that ~ > > somehow brings along. But that seemed to be alright for its use in > > io:format in the first place, didn't it? Also, somehow, it negates > > writing code doubly. > > If you're a C programmer, sure, ~ says bitwise negation. > In Lisp, it doesn't. Not even close. And that's where 'format' > comes from. (Possibly via Prolog.) In _Erlang_, the only use > of "~" is in formats, and there isn't the slightest whiff of > negation about it anywhere. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bengt.kleberg@REDACTED Wed May 19 07:59:10 2010 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 19 May 2010 07:59:10 +0200 Subject: [erlang-questions] what is sysI module ? In-Reply-To: References: Message-ID: <1274248750.4976.8.camel@seasc1137.dyn.rnd.as.sw.ericsson.se> Greetings, I think that sysI is part of the Ericsson Integrated Site concept. More on this page: http://www.ericsson.com/ericsson/corpinfo/publications/review/2005_01/files/2005014.pdf provided it is outside of the firewall... bengt On Wed, 2010-05-19 at 05:10 +0200, William Yangzhuwei wrote: > hi there, > > I am a erlang beginner. > > Does anyone know what is sysI module? > > like sysI:spawn_opts(server, [] ). ? > > I cannot find any doc from Internet regarding this. > Anyone can help to figure out? > thanks! > From kostis@REDACTED Wed May 19 08:33:59 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 19 May 2010 09:33:59 +0300 Subject: [erlang-questions] default dialyzer plt file In-Reply-To: References: Message-ID: <4BF38657.3060603@cs.ntua.gr> Gina Hagg wrote: > Can i transfer my default dialyzer.plt file from my local machine to another > machine using the same erlang lib version, or do i have to build one for > each machine locally. The answer is that you can transfer a PLT file between machines, but to get any benefits the two machines have to have identical configurations. Dialyzer checks the consistency of the information in the PLT against the files that were used to create the PLT. If this information is not up-to-date, the PLT needs to be rebuilt. To perform this check, these files have to be in exactly the same place in both machines (they have to have the same filenames) and have the same checksums. Kostis From erlang@REDACTED Wed May 19 09:51:54 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 May 2010 09:51:54 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 2:39 AM, Robert Virding wrote: > On 18 May 2010 11:46, Joe Armstrong wrote: >> ... >> >> There's no point in expressing the concurrency in an algorithm if the >> hardware can't do it anyway >> (or perhaps there is?) > > Yes, there is. If the algorithm is best expressed in a concurrent way > then you should definitely do so, even if the hardware can't do it. > You are saying what you mean. Or it can be a hint to the > implementation, for example using pmap instead of map means that you > feel that the arguments can be evaluated in parallel, it is then up to > the implementation/hardware to do it if possible. > > Robert > Well of course you're right, in a way and wrong. It's actually the old argument about top-down programming in a new guise. In top-down programming you're not supposed to know what the hardware can actually do. But a quick peep under the covers isn't a bad idea if you want to write efficient code. My current problems are about efficiency and not elegance. The elegant solution is not fast enough. There is no apparent natural concurrency to exploit. Sequential code that is not fast enough must be speeded up - so we have to find bits of concurrency in the middle of the sequential code and map it onto exiting hardware (warts and all). Now I guess the high road to do this would be to describe the problem in a super high level and then use correctness preserving transformations with hints as to the pragmatics of the underlying hardware. The problem, here is that we want the solution a year ago, and nobody knows how to walk the high road (well not quite, but this road is also long and winding, more of a trail in the forest than a road) So far explicit reasoning "we have 4 cores, let's special case it like this" is the only approach that yields faster systems - we don't have enough experience to generalise the results and which is worse, each new generation of multi-cores has different properties (in terms of inter-core message passing times etc. ...) We want to make the stuff go faster on the hardware we have, not write elegant SW for hardware that doesn't exists. (( Do both then - Yea )) /Joe From vladdu55@REDACTED Wed May 19 10:12:50 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 19 May 2010 10:12:50 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 09:51, Joe Armstrong wrote: > On Wed, May 19, 2010 at 2:39 AM, Robert Virding wrote: >> On 18 May 2010 11:46, Joe Armstrong wrote: >>> ... >>> >>> There's no point in expressing the concurrency in an algorithm if the >>> hardware can't do it anyway >>> (or perhaps there is?) >> >> Yes, there is. If the algorithm is best expressed in a concurrent way >> then you should definitely do so, even if the hardware can't do it. >> You are saying what you mean. Or it can be a hint to the >> implementation, for example using pmap instead of map means that you >> feel that the arguments can be evaluated in parallel, it is then up to >> the implementation/hardware to do it if possible. > > My current problems are about efficiency and not elegance. The elegant > solution is not > fast enough. There is no apparent natural concurrency to exploit. > Sequential code > that is not fast enough must be speeded up - so we have to find bits > of concurrency in > the middle of the sequential code and map it onto exiting hardware > (warts and all). > ... > So far explicit reasoning "we have 4 cores, let's special case it like > this" is the only approach > that yields faster systems - we don't have enough experience to > generalise the results and > which is worse, each new generation of multi-cores has different > properties (in terms of inter-core > message passing times etc. ...) > > We want to make the stuff go faster on the hardware we have, not write > elegant SW for > hardware that doesn't exists. Please correct me if i misunderstand, but either you have some concurrency to exploit or you don't. It doesn't matter if the hardware has 4 or 44 cores. IMHO, the first step is (as you started to do) to identify concurrency. There might be several overlapping and incompatible alternatives - this is where the current hardware becomes interesting because different solutions will work on it with different efficiency grades. In other words, to "special case for 4 cores" implies that there's something more general available. best regards, Vlad From clist@REDACTED Wed May 19 11:02:06 2010 From: clist@REDACTED (Angel Alvarez) Date: Wed, 19 May 2010 11:02:06 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: <201005191102.06983.clist@uah.es> El Mi?rcoles, 19 de Mayo de 2010 02:39:42 Robert Virding escribi?: > On 18 May 2010 11:46, Joe Armstrong wrote: > > ... > > > > There's no point in expressing the concurrency in an algorithm if the > > hardware can't do it anyway > > (or perhaps there is?) > > Yes, there is. If the algorithm is best expressed in a concurrent way > then you should definitely do so, even if the hardware can't do it. > You are saying what you mean. Or it can be a hint to the > implementation, for example using pmap instead of map means that you > feel that the arguments can be evaluated in parallel, it is then up to > the implementation/hardware to do it if possible. Again the subttle diference "concurrent vs parallel" where people should think about. Always you should able to express concurrency (tasks that begin and end while other tasks are still running) without regard whether the hardware will serialize them, intermix them or run them in parallel. Where the ability to run things in parallel is the key to performance the ability to express things concurrently is the key to design complex systems. I think exposing tasks to the programming enviroment is more important that having constructs that can (not always) run in parallel. > > Robert > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. __________________________________________ Clist UAH a.k.a Angel __________________________________________ Nunca pude estudiar derecho... (El jorobado de Notredame). From ulf.wiger@REDACTED Wed May 19 11:05:37 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 19 May 2010 11:05:37 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> Message-ID: <4BF3A9E1.8090709@erlang-solutions.com> On 05/19/2010 05:30 AM, Richard O'Keefe wrote: > > On May 19, 2010, at 3:11 PM, Eric Newhuis (personal) wrote: > >> I appreciate all your comments. >> >> I still don't like temporary variables like X or whatever because they >> contain no semantic quality and there is a tension between short names >> to make the pattern obvious and longer names for reunderstanding. > > The variable name X has been used in the examples, > because the examples have been free of content. > No-one, as far as I am aware, is arguing for the use of X > _as such_, although it is at least better than _ or ~ . > > This is why I keep on screaming for *real* examples. > I predict that in real examples, we can find a way to express > what we want with meaningful variable names. Personally, I quite often use constructs like open(F) -> case file:open(F, [read]) of {ok, Fd} -> Fd; {error,_} = Error -> erlang:error(Error) end. Matching the error clause that way not only avoids constructing a tuple needlessly, but also signals that I really don't care /why/ it fails in this particular context - except when debugging, which is why I report it. Raising an exception highlights the fact that I /expect/ it to succeed. Perhaps not the best of examples, but it was the one that came to mind. BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From rdoering@REDACTED Wed May 19 13:14:48 2010 From: rdoering@REDACTED (Ralf Doering) Date: Wed, 19 May 2010 13:14:48 +0200 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: (caio ariede's message of "Tue, 11 May 2010 10:09:02 -0300") References: Message-ID: <87eih8jg7r.fsf@netsys-it.de> caio ariede writes: Hello, > Both platforms give me the same error, just alterning the text specifying > the platform ("x86_64-pc-linux-gnu") and the path, on the error below: > > 1> wx:new(). > > =ERROR REPORT==== 11-May-2010::10:01:54 === > WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: > /usr/lib/erlang/lib/wx-0.98.2/priv > ** exception error: {load_driver,"No driver found"} > in function wxe_server:start/0 > in call from wx:new/1 All official Ubuntu builds of erlang simply have wx disabled. If you want to use it, you have to build your own version of erlang. Ralf From rochus@REDACTED Wed May 19 13:53:09 2010 From: rochus@REDACTED (Nicolai Waniek) Date: Wed, 19 May 2010 13:53:09 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: <4BF3D125.4090900@rochus.net> [ forwarding my mail here, as it happened that I did not reply to the list but to Joe direclty :) ] On 05/19/2010 09:51 AM, Joe Armstrong wrote: > Well of course you're right, in a way and wrong. It's actually the old > argument about > top-down programming in a new guise. In top-down programming you're > not supposed to know > what the hardware can actually do. But a quick peep under the covers > isn't a bad idea > if you want to write efficient code. Quite possibly the correct 'solution' (if there is any) of providing a list of different parallelism pattern is in describing the different algorithms or patterns used first in a rather mathematical way, looking at the theoretically possible throughputs or problems that arise and in a second step looking at the drawbacks that arise with the hardware. Though this is a bit unrelated as it is not erlang-centered, here's an example: Though there are lots of parallel algorithms described for shared memory machines [1], the descriptions are mostly theoretical and when it comes down to implementing the algorithm, say for GPGPU, you have severe drawbacks by the concrete underlying hardware. So what happened to me when searching for a GPGPU algorithm to a rather hard problem was that the theoretical view helped me to figure out which already available algorithms would fit and I had an idea on where to look for, but the lack of description of the problems that arise on different hardware solutions put me back some time researching how to do stuff. In conclusion: Think about describing the patterns in two parts (for each pattern): 1. Theoretical Pattern When it comes down to this, every pattern is usually either divide and conquer or task parallelism. It is quite often possible to determine the worst case time the algorithm would require for different theoretical setups. 2. Practical Limitations This part should mention current (as in 'our hardware at the moment is this, we don't know what it will be in 10 years') hardware limitations and quite possible statistics of different problem sizes so that a reader could decide if the parallelism will be the best way for the problem at hand. Describing a parallel pattern should always mention that one has to figure out if the problem will be best solved in parallel. regards, Nicolai [1] http://techreports.lib.berkeley.edu/accessPages/CSD-88-408.html From hm@REDACTED Wed May 19 14:23:01 2010 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Wed, 19 May 2010 14:23:01 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 9:51 AM, Joe Armstrong wrote: > We want to make the stuff go faster on the hardware we have, > not write elegant SW for hardware that doesn't exists. Right now it feels rather good that you did not have these thoughts in the ancient single core days when the concurrency primitives in Erlang was invented. If the main driver for Erlang concepts would have been speed and not elegance, I suppose that the Erlang language would look quite different. ;-) /H?kan --- H?kan Mattsson Tail-f Systems From erlang@REDACTED Wed May 19 15:21:17 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 May 2010 15:21:17 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 10:12 AM, Vlad Dumitrescu wrote: > On Wed, May 19, 2010 at 09:51, Joe Armstrong wrote: >> On Wed, May 19, 2010 at 2:39 AM, Robert Virding wrote: >>> On 18 May 2010 11:46, Joe Armstrong wrote: >>>> ... >>>> >>>> There's no point in expressing the concurrency in an algorithm if the >>>> hardware can't do it anyway >>>> (or perhaps there is?) >>> >>> Yes, there is. If the algorithm is best expressed in a concurrent way >>> then you should definitely do so, even if the hardware can't do it. >>> You are saying what you mean. Or it can be a hint to the >>> implementation, for example using pmap instead of map means that you >>> feel that the arguments can be evaluated in parallel, it is then up to >>> the implementation/hardware to do it if possible. >> >> My current problems are about efficiency and not elegance. The elegant >> solution is not >> fast enough. There is no apparent natural concurrency to exploit. >> Sequential code >> that is not fast enough must be speeded up - so we have to find bits >> of concurrency in >> the middle of the sequential code and map it onto exiting hardware >> (warts and all). >> ... >> So far explicit reasoning "we have 4 cores, let's special case it like >> this" is the only approach >> that yields faster systems - we don't have enough experience to >> generalise the results and >> which is worse, each new generation of multi-cores has different >> properties (in terms of inter-core >> message passing times etc. ...) >> >> We want to make the stuff go faster on the hardware we have, not write >> elegant SW for >> hardware that doesn't exists. > > Please correct me if i misunderstand, but either you have some > concurrency to exploit or you don't. It doesn't matter if the hardware > has 4 or 44 cores. In my case the problem is given, and the hardware is given. It matters very much how many cores there are and their physical characteristics, since I have to map the problem onto a set of concurrent processes that run on the given hardware. > > IMHO, the first step is (as you started to do) to identify > concurrency. Done that - the #concurrent processes is far greater than the number of cores > There might be several overlapping and incompatible > alternatives - this is where the current hardware becomes interesting > because different solutions will work on it with different efficiency > grades. In other words, to "special case for 4 cores" implies that > there's something more general available. In my case it's because I have a 4 core computer. > best regards, > Vlad > From erlang@REDACTED Wed May 19 15:35:22 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 May 2010 15:35:22 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: No no no .... Thus spake the great progenitors 1) first make it right 2) then make it fast 3) keep it right while making it fast I've done (1) and it wasn't fast enough - now I'm into 2 and 3. If you point me at a formal and provably correct way of doing 2 and 3 I'll start using it. Right now we're doing 2 and 3 by hand. When we have enough experience of 2 and 3 we might be able to generalise the results. This thread started since I wanted to classify the kind of concurrency problems and put them into tight sub-groups so I could see how they mapped onto the hardware we have today. I have a specific problem in mind, at a top level I can use pipeline concurrency, but that does not solve the problem since some of the bits in the pipeline are bottlenecks - and these bits are not naturally concurrent - they use maps etc. which I could change to pmaps, but this has to be done with great care, since it could easily make matters worse. And as for the good old days I was "obsessed by efficiency" as I am today. I still strive to make everything "as inefficient as possible" (ie as beautiful and clear as possible) subject to the condition that it is "fast enough" - it must of course be "fast enough and no faster". 'trouble is project managers do not know the meaning of the expression "fast enough" /Joe 2010/5/19 H?kan Mattsson : > On Wed, May 19, 2010 at 9:51 AM, Joe Armstrong wrote: > >> We want to make the stuff go faster on the hardware we have, >> not write elegant SW for hardware that doesn't exists. > > Right now it feels rather good that you did not have these > thoughts in the ancient single core days when the concurrency > primitives in Erlang was invented. If the main driver for Erlang > concepts would have been speed and not elegance, I suppose > that the Erlang language would look quite different. ;-) > > /H?kan > --- > H?kan Mattsson > Tail-f Systems > From enewhuis@REDACTED Wed May 19 15:44:58 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Wed, 19 May 2010 08:44:58 -0500 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4BF3A9E1.8090709@erlang-solutions.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> <4BF3A9E1.8090709@erlang-solutions.com> Message-ID: <64129BBB-8F29-47BB-A47A-52DDE8EBE787@gmail.com> Ok, nice. But why not leave {error,_} unmatched? Does erlang:error/1 give better post-mortem? On May 19, 2010, at 4:05 AM, Ulf Wiger wrote: > On 05/19/2010 05:30 AM, Richard O'Keefe wrote: >> >> On May 19, 2010, at 3:11 PM, Eric Newhuis (personal) wrote: >> >>> I appreciate all your comments. >>> >>> I still don't like temporary variables like X or whatever because they >>> contain no semantic quality and there is a tension between short names >>> to make the pattern obvious and longer names for reunderstanding. >> >> The variable name X has been used in the examples, >> because the examples have been free of content. >> No-one, as far as I am aware, is arguing for the use of X >> _as such_, although it is at least better than _ or ~ . >> >> This is why I keep on screaming for *real* examples. >> I predict that in real examples, we can find a way to express >> what we want with meaningful variable names. > > Personally, I quite often use constructs like > > open(F) -> > case file:open(F, [read]) of > {ok, Fd} -> Fd; > {error,_} = Error -> > erlang:error(Error) > end. > > Matching the error clause that way not only avoids constructing > a tuple needlessly, but also signals that I really don't care > /why/ it fails in this particular context - except when debugging, > which is why I report it. Raising an exception > highlights the fact that I /expect/ it to succeed. > > Perhaps not the best of examples, but it was the one that came > to mind. > > BR, > Ulf W > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From erlang@REDACTED Wed May 19 15:50:20 2010 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 May 2010 15:50:20 +0200 Subject: Classification of concurrency patterns In-Reply-To: References: Message-ID: Found these they are looking at a similar problem http://www.drdobbs.com/architecture-and-design/224900184 http://parlab.eecs.berkeley.edu/wiki/patterns/patterns /Joe On Wed, May 12, 2010 at 9:31 AM, Joe Armstrong wrote: > I'm interested in trying to classify and name a number of different > parallelization strategies. > > We often use expressions like MPC "message passing concurrency" and > SPC "shared state concurrency" this is not what I'm interested in. > > I'm interested in classifying and naming the strategies we use to > write parallel code. We could *implement* these strategies using MPC > or SSC. > > I've chosen my own names for these below, I have no idea if there are > standard names for these classifications, this is what I've called them: > > 1) Divide and conquer concurrency > 2) Pipeline concurrency > 3) Map reduce concurrency > 4) Identical Job concurrency > 5) Grid concurrency > 6) Job queue concurrency > > They mean the following: > > 1) Divide and conquer concurrency > > Divide the problem into K jobs which can be performed in parallel. > Perform the jobs > Recombine the results > > 2) Pipeline concurrency > > Divide the problem into K jobs that can be performed in a pipeline. > The output of the first job must be the input of the second and so on. > Run all the steps in parallel > > 3) Map reduce concurrency > > ? ?This is similar to 1). Only the recombination step involves > merging results that > ? have the same Key. > > 4) Identical Job concurrency > > ? ?Here the problem does not have to be split into K parts, we > already have N identical > ? ?jobs to do (identical means jobs that will consume similar resources) > > ? There is no recombination step. There are N results. > > 5) Grid concurrency > > ? ?Divide the problem into N*M identical jobs that can be solved on a > NxM grid of processors. > ? ?Each processor can only talk to its direct neighbors > > 6) Job queue concurrency > > ? ?Divide the problem into a set of named workers (who do jobs). Each > worker has an input > ? ?job queue. Workers consume jobs from their job queues and send the > result to other workers. > > ? ?Note: ?[1] Paul Morrison calls this flow based programming > ? ? ? ? ? ? ? [2] This is AMQP > > > ---- > > > That was my first attempt at naming and classifying these patterns. > > I'd be interested to know alternative names for the same things, and > names of systems that employ > the above strategies, and of course missing strategies. > > If we could name and classify these things then we could start writing > gen_divide_and_conquer > gen_pipe etc. > > /Joe > From enewhuis@REDACTED Wed May 19 15:57:14 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Wed, 19 May 2010 08:57:14 -0500 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: What about leased state concurrency? I think I made that phrase up. I'm thinking Linda and tuplespaces and things like async versions of Javaspaces. ...all having a reduced instruction set. Are these covered in your current list? On May 19, 2010, at 8:50 AM, Joe Armstrong wrote: > Found these they are looking at a similar problem > > http://www.drdobbs.com/architecture-and-design/224900184 > http://parlab.eecs.berkeley.edu/wiki/patterns/patterns > > /Joe > > > On Wed, May 12, 2010 at 9:31 AM, Joe Armstrong wrote: >> I'm interested in trying to classify and name a number of different >> parallelization strategies. >> >> We often use expressions like MPC "message passing concurrency" and >> SPC "shared state concurrency" this is not what I'm interested in. >> >> I'm interested in classifying and naming the strategies we use to >> write parallel code. We could *implement* these strategies using MPC >> or SSC. >> >> I've chosen my own names for these below, I have no idea if there are >> standard names for these classifications, this is what I've called them: >> >> 1) Divide and conquer concurrency >> 2) Pipeline concurrency >> 3) Map reduce concurrency >> 4) Identical Job concurrency >> 5) Grid concurrency >> 6) Job queue concurrency >> >> They mean the following: >> >> 1) Divide and conquer concurrency >> >> Divide the problem into K jobs which can be performed in parallel. >> Perform the jobs >> Recombine the results >> >> 2) Pipeline concurrency >> >> Divide the problem into K jobs that can be performed in a pipeline. >> The output of the first job must be the input of the second and so on. >> Run all the steps in parallel >> >> 3) Map reduce concurrency >> >> This is similar to 1). Only the recombination step involves >> merging results that >> have the same Key. >> >> 4) Identical Job concurrency >> >> Here the problem does not have to be split into K parts, we >> already have N identical >> jobs to do (identical means jobs that will consume similar resources) >> >> There is no recombination step. There are N results. >> >> 5) Grid concurrency >> >> Divide the problem into N*M identical jobs that can be solved on a >> NxM grid of processors. >> Each processor can only talk to its direct neighbors >> >> 6) Job queue concurrency >> >> Divide the problem into a set of named workers (who do jobs). Each >> worker has an input >> job queue. Workers consume jobs from their job queues and send the >> result to other workers. >> >> Note: [1] Paul Morrison calls this flow based programming >> [2] This is AMQP >> >> >> ---- >> >> >> That was my first attempt at naming and classifying these patterns. >> >> I'd be interested to know alternative names for the same things, and >> names of systems that employ >> the above strategies, and of course missing strategies. >> >> If we could name and classify these things then we could start writing >> gen_divide_and_conquer >> gen_pipe etc. >> >> /Joe >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vladdu55@REDACTED Wed May 19 16:07:58 2010 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 19 May 2010 16:07:58 +0200 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 15:21, Joe Armstrong wrote: > In my case the problem is given, and the hardware is given. > It matters very much how many cores there are and their physical > characteristics, > since I have to map the problem onto a set of concurrent processes that > run on the given hardware. I see, I think there are two issues that got mixed up (at least in my mind they did). You started the discussion asking about identifying patterns which are abstract and general. This is different than the latter issue about pmap and number of cores, I think. regards, Vlad From ulf.wiger@REDACTED Wed May 19 16:32:55 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 19 May 2010 16:32:55 +0200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <64129BBB-8F29-47BB-A47A-52DDE8EBE787@gmail.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> <4BF3A9E1.8090709@erlang-solutions.com> <64129BBB-8F29-47BB-A47A-52DDE8EBE787@gmail.com> Message-ID: <4BF3F697.9090203@erlang-solutions.com> On 05/19/2010 03:44 PM, Eric Newhuis (personal) wrote: > Ok, nice. But why not leave {error,_} unmatched? Does erlang:error/1 give better post-mortem? Actually, it should have been erlang:error(Error, [F]), and, yes, then it would give a good indication, including the name of the file that couldn't be opened. The main purpose of the wrapper, i.e. not calling file:open/2 directly, is that I prefer it to raise an exception rather than returning an error tuple. This allows me to write the main program flow as Fd = open(File), rather than case file:open(File, [read]) of {ok, Fd} -> ...; {error, Reason} -> ... end. While I could have written open(F) -> case file:open(F, [read]) of {ok, Fd} -> Fd; Error -> erlang:error(Error, [F]) end. ...that would have lost the info that I know that the function returns {ok, Fd} | {error, Reason}. I want to keep that information in the code, without unnecessarily constructing tuples and binding variables that I don't care about. This is a judgement call. Sometimes you really don't want to match on specifics of the return value(s) that you don't care about, because you don't want code to break needlessly due to changes that were not relevant to the code in question. In this particular pattern, my preference is to do a partial match, in order to convey as much knowledge as possible, partly because the function is ubiquitous and its signature very well understood. BR, Ulf W > > On May 19, 2010, at 4:05 AM, Ulf Wiger wrote: > >> On 05/19/2010 05:30 AM, Richard O'Keefe wrote: >>> >>> On May 19, 2010, at 3:11 PM, Eric Newhuis (personal) wrote: >>> >>>> I appreciate all your comments. >>>> >>>> I still don't like temporary variables like X or whatever because they >>>> contain no semantic quality and there is a tension between short names >>>> to make the pattern obvious and longer names for reunderstanding. >>> >>> The variable name X has been used in the examples, >>> because the examples have been free of content. >>> No-one, as far as I am aware, is arguing for the use of X >>> _as such_, although it is at least better than _ or ~ . >>> >>> This is why I keep on screaming for *real* examples. >>> I predict that in real examples, we can find a way to express >>> what we want with meaningful variable names. >> >> Personally, I quite often use constructs like >> >> open(F) -> >> case file:open(F, [read]) of >> {ok, Fd} -> Fd; >> {error,_} = Error -> >> erlang:error(Error) >> end. >> >> Matching the error clause that way not only avoids constructing >> a tuple needlessly, but also signals that I really don't care >> /why/ it fails in this particular context - except when debugging, >> which is why I report it. Raising an exception >> highlights the fact that I /expect/ it to succeed. >> >> Perhaps not the best of examples, but it was the one that came >> to mind. >> >> BR, >> Ulf W >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From jay@REDACTED Wed May 19 17:29:33 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 19 May 2010 08:29:33 -0700 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: On May 19, 2010, at 7:07 AM, Vlad Dumitrescu wrote: > I see, I think there are two issues that got mixed up (at least in my > mind they did). You started the discussion asking about identifying > patterns which are abstract and general. This is different than the > latter issue about pmap and number of cores, I think. Indeed! I thought the thread was about the theoretical and practical task of identifying and describing concurrent patterns. I had wanted to start a thread on that topic, but wasn't ready to put the thought into it so it was a fortunate occurrence when Joe asked about it. The issues related to the classification scheme are: 1) Identifying and organizing the types of concurrency 2) Expressing the patterns in a standard way so they can be referenced 3) Coming up with illustrative problems and code solutions 4) Identifying the factors which affect the performance of each A second topic cropped up as Joe thought about his problem in relation to all the abstract patterns: > There's no point in expressing the concurrency in an algorithm if the > hardware can't do it anyway > (or perhaps there is?) This question has both theoretical and practical considerations. If you are interested in defining and describing concurrency, then it should be expressed as several models with the tradeoffs in performance that each imply (can it scale up or down, how does it deal with failure, partial success and recovery, is it manageable as a system...); however, if you are interested in raw, practical performance concurrency is not the goal of the problem, but rather one possible solution. There is another side of this and it relates to how one develops software. In the past, we expected compilers and serial performance to improve over time. Code is written and just automatically runs faster with newer technology. Just because we are writing concurrent software, we shouldn't abandon this strategy. Software written for 4 cores today should run fast enough, but I would hope for it to run faster on 8, 16 or more cores if the problem can scale up (it may run faster for the same size problem if the granularity is high, but should definitely run faster for larger problems if you can easily increase the granularity with problem size, which is often done by replicating tasks across nodes as map/reduce does). [By granularity I mean the number of concurrent subtasks, or as the inverse size of computation for each subtask. Think grains of sand getting through an hour glass, rather than pebbles.] Finally, Joe's real motivation is a problem that he can't solve fast enough right now on 4 cores: > I have to map the problem onto a set of concurrent processes that > run on the given hardware. There are several issues tied up in this statement: the need for a solution now, the problem of what to do in a future case of the same dilemma and whether theory helps at all. Practical: I would find as much concurrency as is possible, implement that and measure. Coalesce concurrency where it introduces bottlenecks (too many messages in queue, send as a block or merge two processes to eliminate message queue; parallel map => chunked map => serial map). Iterate measuring and trying alternatives. If all coalesces to a serial algorithm, get a faster serial computer or redefine the problem so that results may emerge from several places in the system rather than after all computation. Another alternative is to pay very special care to the CPU cache usage and reformulate the problem to eliminate cache-line faults in low-level computation. This can have a dramatic effect on performance. It may require using NIFs to drop into C for critical sections where you can improve things 100-fold with proper cache usage. Another approach is to use core-oblivious algorithms which attempt to use as much concurrency as they can discover. There is a similar approach to CPU Cache usage http://en.wikipedia.org/wiki/ Cache_oblivious which essentially does recursive dataset splitting until a size is obtained which fits inside your cache. The reassembly of the data will make maximal use of the cache on various platforms without specifying specific parameters (you make an assumption that, for example, 64 bytes of data will definitely fit in the cache and bottom out with an optimal implementation of the 64- byte case). It sounds like you could use a core-oblivious algorithm with cache- oblivious algorithms in each process, with autodetection (hopefully performance observation rather than poking the hardware, since there may be hyper-threading, etc) causing you to adjust the number of core- processes. Future: Compilers / runtimes should do the work of above. Java Hotspot analyzes execution and reorders instructions. We should be able to detect message queue backups or parallel maps or pipelines that have slow spots and rearrange the code to do something equivalent in a more efficient way. The code should be written once and the runtime should adapt to the hardware as best it can through observation, measurement and tuning in real time. Theory: All this practical work, plus a proper classification reference (even if not truly hierarchical), could provide the basis for a new approach to the VM, new methods of expressing data or processes, and new techniques for implementing concurrency vs. serial execution (I imagine something like an accordion that runs serial when compressed, runs concurrently when stretched and feedback measurements press the keys to change the tune). Nicolai Waniek wrote: > Describing a parallel pattern should always mention that one has to > figure out if the problem will be best solved in parallel. I think a guide to concurrency should be just that. The reader is expected to reference other guides to consider alternatives. The OO Patterns book doesn't ask if the problem can be expressed as OO, it assumes the path chosen and then provides the alternatives. jay From ulf.wiger@REDACTED Wed May 19 19:14:02 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 19 May 2010 19:14:02 +0200 Subject: mini-EEP: erlang:sleep() Message-ID: <4BF41C5A.3070509@erlang-solutions.com> After a brief discussion with Tony Rogvall about the currently very unintuitive and (in our opinion) broken implementation of selective receive in interpreted mode (erl_eval), I decided to have a go at implementing a version that doesn't suck up the whole message queue and re-send it. I will confess that I failed horribly, since I only belatedly recalled that hibernate is useless in this setting (it butchers the call stack), and you need a way to sleep until the next message comes in. So here is how I made it work: Some BEAM ASM code follows: {module, mybif}. %% version = 0 {exports, [{module_info,0},{module_info,1},{sleep,0}]}. {attributes, []}. {labels, 8}. {function, sleep, 0, 2}. {label,1}. {func_info,{atom,mybif},{atom,sleep},0}. {label,2}. {wait,{f,3}}. {label,3}. {move,{atom,ok},{x,0}}. return. ... (This was done by writing a very simple function, compiling it and editing the generated .S file.) When compiled, using compile:file("mybif.S", [from_asm]), this becomes a function, mybif:sleep(), that suspends until a message arrives, and then returns 'ok'. It captures the actually very cool feature of hibernate/3, that you can be told when there is a new message to look at, but without the nasty hangover from having the call stack removed, all catches thrown away, etc. With this function, I could emulate selective receive by using process_info(self(), messages), evaluate the patterns over the list of messages, and then, if a message matches, do a receive on that exact message from the real queue. This way, messages remain in the queue, and you get no nasty races from emptying the queue and refilling it. Message with a timeout is implemented by calling start_timer, which has the needed property of including a unique reference in the timeout message. This allows us to check for a timeout by doing a regular selective receive on that timeout msg. The one thing that couldn't be done through normal means was the sleep() above. I cannot think of any bad side-effects of the function. I have not pushed anything to github, as the mybif.S file is more than a little weird. Thoughts, anyone? BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From dougedmunds@REDACTED Wed May 19 19:27:17 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Wed, 19 May 2010 10:27:17 -0700 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: <87eih8jg7r.fsf@netsys-it.de> References: <87eih8jg7r.fsf@netsys-it.de> Message-ID: <4BF41F75.1000701@gmail.com> > All official Ubuntu builds of erlang simply have wx disabled. If you > want to use it, you have to build your own version of erlang. > > Ralf Who makes that decision, and why? == dae From gli.work@REDACTED Wed May 19 19:48:44 2010 From: gli.work@REDACTED (Ivan Glushkov) Date: Wed, 19 May 2010 21:48:44 +0400 Subject: regular expression - exponential complexity Message-ID: Hi. I'm almost sure that you know this, so JFYI. I found regexp that has exponential complexity while in perl the same regexp seems to grow linearily. So the question is, are you going to move to re2 or any other DFA regexp library? details: cat test.erl -module(test). -export([ str/1, f/1, fc/1 ]). str(Len) -> "exmample." ++ string:chars($c,Len) ++ "?a". f(Len) -> re:run(str(Len),"^([a-z0-9\\-\\_]+\\.)*[a-z0-9]([a-z0-9\\-]*[a-z0-9])?\\.[a-z]([a-z0-9\\-]*[a-z0-9])+$", [{capture, none}]). fc(Len) -> {ok, Re} = re:compile("^([a-z0-9\\-\\_]+\\.)*[a-z0-9]([a-z0-9\\-]*[a-z0-9])?\\.[a-z]([a-z0-9\\-]*[a-z0-9])+$"), re:run(str(Len), Re, [{capture, none}]). Some results (f and fc give the same result). string length / frequency (run cycles per second) 3 50000 5 30000 10 1455 11 1136 12 586 13 292 14 146 15 73 16 37 Ivan. From seanhern@REDACTED Wed May 19 21:34:29 2010 From: seanhern@REDACTED (Sean Hernandez) Date: Wed, 19 May 2010 14:34:29 -0500 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: > Another approach is to use core-oblivious algorithms which attempt to use as much concurrency as they can discover. > Future: > Compilers / runtimes should do the work of above. Java Hotspot analyzes execution and reorders instructions. We should be able to detect message queue backups or parallel maps or pipelines that have slow spots and rearrange the code to do something equivalent in a more efficient way. The code should be written once and the runtime should adapt to the hardware as best it can through observation, measurement and tuning in real time. I think there really separate things: (1) discoverying a machine's processor, core and memory topology and (2) scheduling instructions within a core. Discovering a machine's processor, core and memory topology is important because of the asymmetry of latencies when accessing memory that is local to a core or processor vs. memory that is remote (i.e. NUMA). When you've minimized the probability of accessing remote memory. Now you can minimize instruction cache misses (can't do anything for the data cache) using an opportunistic scheduler that "reuses" instructions that has previously executed (resulting in out-of-order execution). These are really about (1) discovering the core topology of the machine the runtime is executing on taking into account latencies, (2) affinitizing schedulers for On Wed, May 19, 2010 at 10:29 AM, Jay Nelson wrote: > > On May 19, 2010, at 7:07 AM, Vlad Dumitrescu wrote: > >> I see, I think there are two issues that got mixed up (at least in my >> mind they did). You started the discussion asking about identifying >> patterns which are abstract and general. This is different than the >> latter issue about pmap and number of cores, I think. > > Indeed! > > I thought the thread was about the theoretical and practical task of identifying and describing concurrent patterns. ?I had wanted to start a thread on that topic, but wasn't ready to put the thought into it so it was a fortunate occurrence when Joe asked about it. ?The issues related to the classification scheme are: > > 1) Identifying and organizing the types of concurrency > 2) Expressing the patterns in a standard way so they can be referenced > 3) Coming up with illustrative problems and code solutions > 4) Identifying the factors which affect the performance of each > > A second topic cropped up as Joe thought about his problem in relation to all the abstract patterns: > > > There's no point in expressing the concurrency in an algorithm if the > > hardware can't do it anyway > > (or perhaps there is?) > > This question has both theoretical and practical considerations. ?If you are interested in defining and describing concurrency, then it should be expressed as several models with the tradeoffs in performance that each imply (can it scale up or down, how does it deal with failure, partial success and recovery, is it manageable as a system...); however, if you are interested in raw, practical performance concurrency is not the goal of the problem, but rather one possible solution. > > There is another side of this and it relates to how one develops software. ?In the past, we expected compilers and serial performance to improve over time. ?Code is written and just automatically runs faster with newer technology. ?Just because we are writing concurrent software, we shouldn't abandon this strategy. ?Software written for 4 cores today should run fast enough, but I would hope for it to run faster on 8, 16 or more cores if the problem can scale up (it may run faster for the same size problem if the granularity is high, but should definitely run faster for larger problems if you can easily increase the granularity with problem size, which is often done by replicating tasks across nodes as map/reduce does). > > [By granularity I mean the number of concurrent subtasks, or as the inverse size of computation for each subtask. ?Think grains of sand getting through an hour glass, rather than pebbles.] > > Finally, Joe's real motivation is a problem that he can't solve fast enough right now on 4 cores: > > > I have to map the problem onto a set of concurrent processes that > > run on the given hardware. > > There are several issues tied up in this statement: the need for a solution now, the problem of what to do in a future case of the same dilemma and whether theory helps at all. > > Practical: > > I would find as much concurrency as is possible, implement that and measure. ?Coalesce concurrency where it introduces bottlenecks (too many messages in queue, send as a block or merge two processes to eliminate message queue; parallel map => chunked map => serial map). ?Iterate measuring and trying alternatives. > > If all coalesces to a serial algorithm, get a faster serial computer or redefine the problem so that results may emerge from several places in the system rather than after all computation. ?Another alternative is to pay very special care to the CPU cache usage and reformulate the problem to eliminate cache-line faults in low-level computation. ?This can have a dramatic effect on performance. ?It may require using NIFs to drop into C for critical sections where you can improve things 100-fold with proper cache usage. > > Another approach is to use core-oblivious algorithms which attempt to use as much concurrency as they can discover. ?There is a similar approach to CPU Cache usage http://en.wikipedia.org/wiki/Cache_oblivious which essentially does recursive dataset splitting until a size is obtained which fits inside your cache. ?The reassembly of the data will make maximal use of the cache on various platforms without specifying specific parameters (you make an assumption that, for example, 64 bytes of data will definitely fit in the cache and bottom out with an optimal implementation of the 64-byte case). > > It sounds like you could use a core-oblivious algorithm with cache-oblivious algorithms in each process, with autodetection (hopefully performance observation rather than poking the hardware, since there may be hyper-threading, etc) causing you to adjust the number of core-processes. > > Future: > > Compilers / runtimes should do the work of above. ?Java Hotspot analyzes execution and reorders instructions. ?We should be able to detect message queue backups or parallel maps or pipelines that have slow spots and rearrange the code to do something equivalent in a more efficient way. ?The code should be written once and the runtime should adapt to the hardware as best it can through observation, measurement and tuning in real time. > > Theory: > > All this practical work, plus a proper classification reference (even if not truly hierarchical), could provide the basis for a new approach to the VM, new methods of expressing data or processes, and new techniques for implementing concurrency vs. serial execution (I imagine something like an accordion that runs serial when compressed, runs concurrently when stretched and feedback measurements press the keys to change the tune). > > > Nicolai Waniek wrote: > > > Describing a parallel pattern should always mention that one has to > > figure out if the problem will be best solved in parallel. > > I think a guide to concurrency should be just that. ?The reader is expected to reference other guides to consider alternatives. ?The OO Patterns book doesn't ask if the problem can be expressed as OO, it assumes the path chosen and then provides the alternatives. > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From seanhern@REDACTED Wed May 19 22:17:22 2010 From: seanhern@REDACTED (Sean Hernandez) Date: Wed, 19 May 2010 15:17:22 -0500 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: But in order to do adaptive execution a model of your execution environment needs to be built (e.g. processor, core, memory (this includes cache sizes too btw), execution latencies, queue lengths, similarity of instructions (instruction caches misses, etc) up so that adaptation algorithms can figure out what to tweak. Maybe these are the elements of the provably correct model that Joe is looking for? On Wed, May 19, 2010 at 2:56 PM, Jay Nelson wrote: >> These are really about (1) discovering the core topology of the >> machine the runtime is executing on taking into account latencies, (2) >> affinitizing schedulers for > > An alternative is to watch the execution, reducing the data set size, until > a speed up is seen. ?In this way you don't optimize based on how it _should_ > work, but on how the system is actually performing. ?Once the optimal data > set size is determined, the data can be automatically chopped up > accordingly. > > Better to detect cache (or who knows what future technology) speed up so > that the code is automatically adaptive on alternative hardware. ?Likewise, > you could build a large queue of tasks, then spawn new workers periodically > measuring number of tasks / unit of time. ?At some threshold the overhead > will show up and slow down progress, so throttle back the number of open > processes in the worker pool, dynamically deciding based performance rather > than number of cores, or schedulers or what have you. > > jay > > From caio.ariede@REDACTED Wed May 19 22:28:36 2010 From: caio.ariede@REDACTED (caio ariede) Date: Wed, 19 May 2010 17:28:36 -0300 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: <4BF41F75.1000701@gmail.com> References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: I know this. I think that they removed Wx support because it's dependencies. I can run wx:module_info() on both systems (mac and linux), but when I run wx:new() I get the same error: =ERROR REPORT==== 11-May-2010::10:01:54 === WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: /usr/lib/erlang/lib/wx-0.98.2/ priv ** exception error: {load_driver,"No driver found"} in function wxe_server:start/0 in call from wx:new/1 Caio Ariede http://caioariede.com/ On Wed, May 19, 2010 at 2:27 PM, Doug Edmunds (gmail) wrote: > All official Ubuntu builds of erlang simply have wx disabled. If you >> want to use it, you have to build your own version of erlang. >> >> Ralf >> > > Who makes that decision, and why? > > == dae > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuncer.ayaz@REDACTED Wed May 19 22:38:04 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 19 May 2010 22:38:04 +0200 Subject: [erlang-questions] regular expression - exponential complexity In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 7:48 PM, Ivan Glushkov wrote: > Hi. > > I'm almost sure that you know this, so JFYI. > I found regexp that has exponential complexity while in perl > the same regexp seems to grow linearily. > > So the question is, are you going to move to re2 or any other > DFA regexp library? [snip] > Some results (f and fc give the same result). > > string length / frequency (run cycles per second) > 3 ? ? ? 50000 > 5 ? ? ? 30000 > 10 ? ? ?1455 > 11 ? ? ?1136 > 12 ? ? ?586 > 13 ? ? ?292 > 14 ? ? ?146 > 15 ? ? ?73 > 16 ? ? ?37 Have you tried the same within a vanilla pcretest shell invoked as 'pcretest -dfa'? OTP's PCRE is patched for restartability/interruptability reasons and also compiled with the NO_RECURSE/--disable-stack-for-recursion option. The PCRE docs say this option has a speed impact on pcre_exec() but not on pcre_dfa_exec(). As re seems to strive for Perl compatibility erts_pcre makes use of erts_pcre_exec() only. The erts_ prefix is presumably used to reflect the patched nature of the bundled PCRE. If you're curious you might write and test a linked-in driver/nif with stock PCRE :). Actually the comments in pcre_dfa_exec() say that it's using a 'sort of DFA algo' and not 'a true FSM'. Let's document that info bit here before anyone asks. See the pcrestack(3) and pcretest(1) manpages for details. As you have mentioned it: If the feature set of RE2 suits your needs have a look at http://github.com/tuncer/re2. From rvirding@REDACTED Wed May 19 22:47:33 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 19 May 2010 22:47:33 +0200 Subject: [erlang-questions] regular expression - exponential complexity In-Reply-To: References: Message-ID: For a more in depth description of the problem see the paper by Russ Cox: http://swtch.com/~rsc/regexp/regexp1.html Even better read his page on regexps, http://swtch.com/~rsc/regexp/ , which describes more solutions and how re2 is implemented. Robert On 19 May 2010 19:48, Ivan Glushkov wrote: > Hi. > > I'm almost sure that you know this, so JFYI. > I found regexp that has exponential complexity while in perl the same > regexp seems to grow linearily. > > So the question is, are you going to move to re2 or any other ?DFA > regexp library? > > details: > > cat test.erl > -module(test). > -export([ > ? ? ? ?str/1, > ? ? ? ?f/1, > ? ? ? ?fc/1 > ? ?]). > > str(Len) -> "exmample." ++ string:chars($c,Len) ++ "?a". > > f(Len) -> > ? ?re:run(str(Len),"^([a-z0-9\\-\\_]+\\.)*[a-z0-9]([a-z0-9\\-]*[a-z0-9])?\\.[a-z]([a-z0-9\\-]*[a-z0-9])+$", > [{capture, none}]). > > fc(Len) -> > ? ?{ok, Re} = re:compile("^([a-z0-9\\-\\_]+\\.)*[a-z0-9]([a-z0-9\\-]*[a-z0-9])?\\.[a-z]([a-z0-9\\-]*[a-z0-9])+$"), > ? ?re:run(str(Len), Re, [{capture, none}]). > > > Some results (f and fc give the same result). > > string length / frequency (run cycles per second) > 3 ? ? ? 50000 > 5 ? ? ? 30000 > 10 ? ? ?1455 > 11 ? ? ?1136 > 12 ? ? ?586 > 13 ? ? ?292 > 14 ? ? ?146 > 15 ? ? ?73 > 16 ? ? ?37 > > > Ivan. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jay@REDACTED Wed May 19 21:56:06 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 19 May 2010 12:56:06 -0700 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: > These are really about (1) discovering the core topology of the > machine the runtime is executing on taking into account latencies, (2) > affinitizing schedulers for An alternative is to watch the execution, reducing the data set size, until a speed up is seen. In this way you don't optimize based on how it _should_ work, but on how the system is actually performing. Once the optimal data set size is determined, the data can be automatically chopped up accordingly. Better to detect cache (or who knows what future technology) speed up so that the code is automatically adaptive on alternative hardware. Likewise, you could build a large queue of tasks, then spawn new workers periodically measuring number of tasks / unit of time. At some threshold the overhead will show up and slow down progress, so throttle back the number of open processes in the worker pool, dynamically deciding based performance rather than number of cores, or schedulers or what have you. jay From dougedmunds@REDACTED Wed May 19 23:06:28 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Wed, 19 May 2010 14:06:28 -0700 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: <4BF452D4.3090607@gmail.com> I have Sun's VBox on an XP host (x32), running a Fedora Linux guest. Fedora has an add-on rpm for Erlang which is labeled "A library for wxWidgets support in Erlang" erlang-xmerl-R13B-04.11.fc12 (i686) That rpm includes a compiled driver (which is missing in Ubuntu). It gets installed in: /usr/lib/erlang/lib/wx-0.98.5/priv/i386-redhat-linux-gnu/wxe_driver.so I doubt that the RedHat driver will work in Ubuntu, however. On 5/19/2010 1:28 PM, caio ariede wrote: > I know this. I think that they removed Wx support because it's dependencies. > > I can run wx:module_info() on both systems (mac and linux), but when I run > wx:new() I get the same error: > > =ERROR REPORT==== 11-May-2010::10:01:54 === > WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: > /usr/lib/erlang/lib/wx-0.98.2/ > priv > ** exception error: {load_driver,"No driver found"} > in function wxe_server:start/0 > in call from wx:new/1 > > Caio Ariede > http://caioariede.com/ > > > On Wed, May 19, 2010 at 2:27 PM, Doug Edmunds (gmail)> wrote: > >> All official Ubuntu builds of erlang simply have wx disabled. If you >>> want to use it, you have to build your own version of erlang. >>> >>> Ralf >>> >> >> Who makes that decision, and why? >> >> == dae >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From tony@REDACTED Wed May 19 22:15:10 2010 From: tony@REDACTED (Tony Rogvall) Date: Wed, 19 May 2010 22:15:10 +0200 Subject: [erlang-questions] mini-EEP: erlang:sleep() In-Reply-To: <4BF41C5A.3070509@erlang-solutions.com> References: <4BF41C5A.3070509@erlang-solutions.com> Message-ID: <0AD590CB-BEE6-4E2E-B3BB-86E1FF699CD3@rogvall.se> You see, it was not that hard to write beam asm code ;-) Way to go .... maybe it's time to get some bif's doing this properly ? No disrespect ! /Tony On 19 maj 2010, at 19.14, Ulf Wiger wrote: > After a brief discussion with Tony Rogvall about the currently > very unintuitive and (in our opinion) broken implementation of > selective receive in interpreted mode (erl_eval), I decided to > have a go at implementing a version that doesn't suck up the whole > message queue and re-send it. > > I will confess that I failed horribly, since I only belatedly > recalled that hibernate is useless in this setting (it butchers > the call stack), and you need a way to sleep until the next message > comes in. > > So here is how I made it work: > > Some BEAM ASM code follows: > > {module, mybif}. %% version = 0 > > {exports, [{module_info,0},{module_info,1},{sleep,0}]}. > > {attributes, []}. > > {labels, 8}. > > > {function, sleep, 0, 2}. > {label,1}. > {func_info,{atom,mybif},{atom,sleep},0}. > {label,2}. > {wait,{f,3}}. > {label,3}. > {move,{atom,ok},{x,0}}. > return. > ... > > (This was done by writing a very simple function, compiling it > and editing the generated .S file.) > > When compiled, using compile:file("mybif.S", [from_asm]), this > becomes a function, mybif:sleep(), that suspends until a message > arrives, and then returns 'ok'. It captures the actually very cool > feature of hibernate/3, that you can be told when there is a new > message to look at, but without the nasty hangover from having the > call stack removed, all catches thrown away, etc. > > With this function, I could emulate selective receive by using > process_info(self(), messages), evaluate the patterns over the > list of messages, and then, if a message matches, do a receive on > that exact message from the real queue. This way, messages remain > in the queue, and you get no nasty races from emptying the queue > and refilling it. > > Message with a timeout is implemented by calling start_timer, > which has the needed property of including a unique reference > in the timeout message. This allows us to check for a timeout > by doing a regular selective receive on that timeout msg. > > The one thing that couldn't be done through normal means was > the sleep() above. I cannot think of any bad side-effects of > the function. > > I have not pushed anything to github, as the mybif.S file is more > than a little weird. > > Thoughts, anyone? > > BR, > Ulf W > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From rvirding@REDACTED Thu May 20 00:35:07 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 20 May 2010 00:35:07 +0200 Subject: [erlang-questions] mini-EEP: erlang:sleep() In-Reply-To: <4BF41C5A.3070509@erlang-solutions.com> References: <4BF41C5A.3070509@erlang-solutions.com> Message-ID: IIRC then way way back when we had an option to write Erlang functions in modules directly in BEAM (or was it JAM code? ). It was something like: -asm(sleep, 0, [...]). For "real" programming. Robert On 19 May 2010 19:14, Ulf Wiger wrote: > After a brief discussion with Tony Rogvall about the currently > very unintuitive and (in our opinion) broken implementation of > selective receive in interpreted mode (erl_eval), I decided to > have a go at implementing a version that doesn't suck up the whole > message queue and re-send it. > > I will confess that I failed horribly, since I only belatedly > recalled that hibernate is useless in this setting (it butchers > the call stack), and you need a way to sleep until the next message > comes in. > > So here is how I made it work: > > Some BEAM ASM code follows: > > {module, mybif}. ?%% version = 0 > > {exports, [{module_info,0},{module_info,1},{sleep,0}]}. > > {attributes, []}. > > {labels, 8}. > > > {function, sleep, 0, 2}. > ?{label,1}. > ? ?{func_info,{atom,mybif},{atom,sleep},0}. > ?{label,2}. > ? ?{wait,{f,3}}. > ?{label,3}. > ? ?{move,{atom,ok},{x,0}}. > ? ?return. > ... > > (This was done by writing a very simple function, compiling it > and editing the generated .S file.) > > When compiled, using compile:file("mybif.S", [from_asm]), this > becomes a function, mybif:sleep(), that suspends until a message > arrives, and then returns 'ok'. It captures the actually very cool > feature of hibernate/3, that you can be told when there is a new > message to look at, but without the nasty hangover from having the > call stack removed, all catches thrown away, etc. > > With this function, I could emulate selective receive by using > process_info(self(), messages), evaluate the patterns over the > list of messages, and then, if a message matches, do a receive on > that exact message from the real queue. This way, messages remain > in the queue, and you get no nasty races from emptying the queue > and refilling it. > > Message with a timeout is implemented by calling start_timer, > which has the needed property of including a unique reference > in the timeout message. This allows us to check for a timeout > by doing a regular selective receive on that timeout msg. > > The one thing that couldn't be done through normal means was > the sleep() above. I cannot think of any bad side-effects of > the function. > > I have not pushed anything to github, as the mybif.S file is more > than a little weird. > > Thoughts, anyone? > > BR, > Ulf W > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jay@REDACTED Thu May 20 00:07:42 2010 From: jay@REDACTED (Jay Nelson) Date: Wed, 19 May 2010 15:07:42 -0700 Subject: [erlang-questions] Re: Classification of concurrency patterns In-Reply-To: References: Message-ID: <35DAC1F4-CFEC-4EB3-A2C3-312138A286C8@duomark.com> On May 19, 2010, at 1:17 PM, Sean Hernandez wrote: > But in order to do adaptive execution a model of your execution > environment needs to be built (e.g. processor, core, memory (this > includes cache sizes too btw), execution latencies, queue lengths, > similarity of instructions (instruction caches misses, etc) up so that > adaptation algorithms can figure out what to tweak. If you are just copying data from one array to another, you can tell when a cache boundary is tripped by the slowdown in the performance. If you write progressively more data, measuring the time lapse, you can find the array size that first causes the slowdown. You don't need to do any complicated modeling. Then you can make your divide and conquer algorithm choose the maximum size for each of the divide elements to get maximal performance. As I said earlier, the same thing can be done with a task queue and a worker pool. If there are 1M tasks over an hour, it is easy to adjust the number of workers every 10 seconds for the first 10 minutes to get a feel for performance. Then choose the optimal number of workers for the rest of the time. You just need to have elements of your algorithm that can be tuned (e.g., number of processes, data sequence size in divide and conquer, message batch size, etc). jay From ok@REDACTED Thu May 20 01:37:23 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 20 May 2010 11:37:23 +1200 Subject: [erlang-questions] I think I wish I could write case Any of whatever -> _ end. In-Reply-To: <4BF3A9E1.8090709@erlang-solutions.com> References: <09398399-7080-4902-ACBC-7DD2B53AB198@gmail.com> <3AE5501F-4F7B-4C33-9842-A771C114D027@cs.otago.ac.nz> <4824C305-02CF-41C3-A66F-52E20DE16999@gmail.com> <1521FB5D-6B92-431C-B179-C6CD62A5A987@cs.otago.ac.nz> <785C50B0-4468-4D71-8D1F-A60837D87104@cs.otago.ac.nz> <4BF3A9E1.8090709@erlang-solutions.com> Message-ID: <98927DBE-A469-42BD-85D1-0A89CBF5FEB6@cs.otago.ac.nz> On May 19, 2010, at 9:05 PM, Ulf Wiger wrote: a completely useless example for this thread, because it is an example of someone doing something RIGHT. > open(F) -> > case file:open(F, [read]) of > {ok, Fd} -> Fd; > {error,_} = Error -> > erlang:error(Error) > end. The one possible improvement would be ok({ok,Good_Result}) -> Good_Result; ok({error,Reason}) -> erlang:error(Reason). open(F) -> ok(file:open(F, [read])). possibly extending ok/1 with additional clauses for additional forms of good and bad outcome. > From ulf.wiger@REDACTED Thu May 20 10:02:42 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 20 May 2010 10:02:42 +0200 Subject: [erlang-questions] mini-EEP: erlang:sleep() In-Reply-To: <0AD590CB-BEE6-4E2E-B3BB-86E1FF699CD3@rogvall.se> References: <4BF41C5A.3070509@erlang-solutions.com> <0AD590CB-BEE6-4E2E-B3BB-86E1FF699CD3@rogvall.se> Message-ID: <4BF4ECA2.6060405@erlang-solutions.com> On 05/19/2010 10:15 PM, Tony Rogvall wrote: > You see, it was not that hard to write beam asm code ;-) > Way to go .... maybe it's time to get some bif's doing this properly ? > No disrespect ! Indeed. Before going to sleep, I spotted the 'obvious' race condition, so this solution is inadequate. :) I think my implementation is still better than the existing one, but it should probably only serve as a basis for discussion on how to solve it properly. BR, Ulf W > > > /Tony > > On 19 maj 2010, at 19.14, Ulf Wiger wrote: > >> After a brief discussion with Tony Rogvall about the currently >> very unintuitive and (in our opinion) broken implementation of >> selective receive in interpreted mode (erl_eval), I decided to >> have a go at implementing a version that doesn't suck up the whole >> message queue and re-send it. >> >> I will confess that I failed horribly, since I only belatedly >> recalled that hibernate is useless in this setting (it butchers >> the call stack), and you need a way to sleep until the next message >> comes in. >> >> So here is how I made it work: >> >> Some BEAM ASM code follows: >> >> {module, mybif}. %% version = 0 >> >> {exports, [{module_info,0},{module_info,1},{sleep,0}]}. >> >> {attributes, []}. >> >> {labels, 8}. >> >> >> {function, sleep, 0, 2}. >> {label,1}. >> {func_info,{atom,mybif},{atom,sleep},0}. >> {label,2}. >> {wait,{f,3}}. >> {label,3}. >> {move,{atom,ok},{x,0}}. >> return. >> ... >> >> (This was done by writing a very simple function, compiling it >> and editing the generated .S file.) >> >> When compiled, using compile:file("mybif.S", [from_asm]), this >> becomes a function, mybif:sleep(), that suspends until a message >> arrives, and then returns 'ok'. It captures the actually very cool >> feature of hibernate/3, that you can be told when there is a new >> message to look at, but without the nasty hangover from having the >> call stack removed, all catches thrown away, etc. >> >> With this function, I could emulate selective receive by using >> process_info(self(), messages), evaluate the patterns over the >> list of messages, and then, if a message matches, do a receive on >> that exact message from the real queue. This way, messages remain >> in the queue, and you get no nasty races from emptying the queue >> and refilling it. >> >> Message with a timeout is implemented by calling start_timer, >> which has the needed property of including a unique reference >> in the timeout message. This allows us to check for a timeout >> by doing a regular selective receive on that timeout msg. >> >> The one thing that couldn't be done through normal means was >> the sleep() above. I cannot think of any bad side-effects of >> the function. >> >> I have not pushed anything to github, as the mybif.S file is more >> than a little weird. >> >> Thoughts, anyone? >> >> BR, >> Ulf W >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From kyleqian@REDACTED Thu May 20 11:07:43 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Thu, 20 May 2010 17:07:43 +0800 Subject: About open_port and epipe error Message-ID: Hi, I use open_port to run .exe file under windows xp: Port = open_port({spawn, "./algorithm/algorun"}, [{packet, 2}]), link(Port), Port ! {self(), {command, TaskContent}}, I run this executable file 5 times, and it do nothing but only show a message box(do not read/write anything). I find that when they stop normally( I click the message box, and return 0 in main function), only the first one make my erlang module receiving {'EXIT', Port, normal}, the others are {'EXIT', Port, epipe}. So why I receive this epipe error? I am using Erlang R13B04 (erts-5.7.5). Thanks! From Mike.French@REDACTED Thu May 20 12:06:13 2010 From: Mike.French@REDACTED (French, Mike) Date: Thu, 20 May 2010 11:06:13 +0100 Subject: [erlang-questions] Re: Classification of concurrency patterns Message-ID: <3F8EEA01CF53D74DB4A9EC314D82B4F3224736@wells154942> Also see this work on nested data parallelism for Haskell: http://haskell.org/haskellwiki/GHC/Data_Parallel_Haskell there are two good presentations linked in the Further Reading section. Mike > -----Original Message----- > From: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED]On > Behalf Of Joe Armstrong > Sent: 19 May 2010 14:50 > To: Erlang > Subject: [erlang-questions] Re: Classification of concurrency patterns > > > Found these they are looking at a similar problem > > http://www.drdobbs.com/architecture-and-design/224900184 > http://parlab.eecs.berkeley.edu/wiki/patterns/patterns > > /Joe Thales UK Ltd (Wells) DISCLAIMER: The information contained in this e-mail is confidential. It may also be legally privileged. It is intended only for the stated addressee(s) and access to it by any other person is unauthorised. If you are not an addressee, you must not disclose, copy, circulate or in any other way use or rely on the information contained in this e-mail. Such unauthorised use may be unlawful. We may monitor all e-mail communications through our networks. If you have received this e-mail in error, please inform us immediately on sender's telephone number above and delete it and all copies from your system. We accept no responsibility for changes to any e-mail which occur after it has been sent. Attachments to this e-mail may contain software viruses which could damage your system. We therefore recommend you virus-check all attachments before opening. Thales UK Ltd. Registered Office: 2 Dashwood Lang Road, The Bourne Business Park, Addlestone, Weybridge, Surrey KT15 2NX Registered in England No. 868273 From mononcqc@REDACTED Thu May 20 14:11:43 2010 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 20 May 2010 08:11:43 -0400 Subject: [erlang-questions] About open_port and epipe error In-Reply-To: References: Message-ID: Have you tried with werl.exe or erl.exe? As far As I know, there are some limitations to what werl can do with regards to redirecting i/o, which could be the source of the problem here. On Thu, May 20, 2010 at 5:07 AM, ??? wrote: > Hi, I use open_port to run .exe file under windows xp: > Port = open_port({spawn, "./algorithm/algorun"}, [{packet, 2}]), > link(Port), > Port ! {self(), {command, TaskContent}}, > I run this executable file 5 times, and it do nothing but only show a > message box(do not read/write anything). I find that when they stop > normally( I click the message box, and return 0 in main function), only the > first one make my erlang module receiving {'EXIT', Port, normal}, the > others are {'EXIT', Port, epipe}. > > So why I receive this epipe error? I am using Erlang R13B04 (erts-5.7.5). > > Thanks! > From tuncer.ayaz@REDACTED Thu May 20 17:29:52 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 20 May 2010 17:29:52 +0200 Subject: [erlang-questions] is EEP 31 implemented in R13B04 ? In-Reply-To: References: Message-ID: On Sat, May 1, 2010 at 9:44 AM, mabrek wrote: > Hello, > > http://www.erlang.org/eeps/eep-0031.html#reference-implementation > states that it will be implemented in R13B04, but I can't find it in > my distribution (ubuntu packages from ppa > https://edge.launchpad.net/~erlang-dev/+archive/ppa ) > Is it implemented in R13B04 or how else can I find the implementation? EEP-0031 has today been merged to the dev branch for inclusion in R14. Thanks Patrik and OTP team! From MARTIN.LOGAN@REDACTED Thu May 20 17:34:37 2010 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Thu, 20 May 2010 10:34:37 -0500 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: Message-ID: We are working hard on it still. We have a secure website developed and are holding back on releasing it right now because of some other refactoring that is going on. Erlware should have most of the important code already stored for most versions of Erts. All the code comes with its src so you are free to rebuild it if you want. The caveat is that Erlware repos are not secure. They are quite open right now. The community is small enough such that we don't have problems. That said, when we finally push out our new site and repo that will change. It is also possible to host your own internal repo easily with a project called portius. Here is a link to a video explaining how to do it. http://www.youtube.com/watch?v=ogbz0Ram1_w Cheers, Martin -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Steve Davis Sent: Monday, May 17, 2010 8:23 PM To: erlang-questions@REDACTED Subject: [erlang-questions] Re: Library management in erlang. Closest I have seen are: http://projects.trapexit.org/web/ http://jungerl.sourceforge.net/ http://erlware.org/ /s On May 17, 7:35?pm, Tim Uckun wrote: > Is there anything equivalent to ruby gems in erlang? > > I am trying to follow a couple of tutorials they all say "go download > this library, git clone this library" etc. ?How do you guys keep your > libraries current and secure? is there an equivalent of CPAN or > rubyforge where I can search for libraries that might be of use? > > Are there the equivalent of ?gem search, gem list, gem install, gem > update,gem uninstall etc. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > -- > You received this message because you are subscribed to the Google Groups "Erlang Programming" group. > To post to this group, send email to erlang-programming@REDACTED > To unsubscribe from this group, send email to erlang-programming+unsubscribe@REDACTED > For more options, visit this group athttp://groups.google.com/group/erlang-programming?hl=en. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From igorrs@REDACTED Thu May 20 17:53:28 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Thu, 20 May 2010 12:53:28 -0300 Subject: Some tips on Mnesia Message-ID: For those people who are starting to work with Mnesia, I'm writing a short sequence of blog posts that are intended to point out some relevant facts that don't seem to be well-known and could hurt you in the beginning. I'll try to focus on the information that you can't google. ;-) But maybe in the end I'll also summarize the important tips that I learned from other blogs. Here is the first post: http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html Good luck. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From gershon.bialer@REDACTED Thu May 20 22:07:18 2010 From: gershon.bialer@REDACTED (gershon bialer) Date: Thu, 20 May 2010 15:07:18 -0500 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: Message-ID: Does rebar have a package management system based off github and mercurial archives? I've heard something to that effect, but I have yet to see how that actually works. On Thu, May 20, 2010 at 10:34 AM, Logan, Martin wrote: > We are working hard on it still. We have a secure website developed and are > holding back on releasing it right now because of some other refactoring > that is going on. Erlware should have most of the important code already > stored for most versions of Erts. All the code comes with its src so you are > free to rebuild it if you want. The caveat is that Erlware repos are not > secure. They are quite open right now. The community is small enough such > that we don't have problems. That said, when we finally push out our new > site and repo that will change. > > It is also possible to host your own internal repo easily with a project > called portius. Here is a link to a video explaining how to do it. > http://www.youtube.com/watch?v=ogbz0Ram1_w > > Cheers, > Martin > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Steve Davis > Sent: Monday, May 17, 2010 8:23 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] Re: Library management in erlang. > > Closest I have seen are: > > http://projects.trapexit.org/web/ > > http://jungerl.sourceforge.net/ > > http://erlware.org/ > > /s > > On May 17, 7:35 pm, Tim Uckun wrote: > > Is there anything equivalent to ruby gems in erlang? > > > > I am trying to follow a couple of tutorials they all say "go download > > this library, git clone this library" etc. How do you guys keep your > > libraries current and secure? is there an equivalent of CPAN or > > rubyforge where I can search for libraries that might be of use? > > > > Are there the equivalent of gem search, gem list, gem install, gem > > update,gem uninstall etc. > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > > > -- > > You received this message because you are subscribed to the Google Groups > "Erlang Programming" group. > > To post to this group, send email to erlang-programming@REDACTED > . > > To unsubscribe from this group, send email to > erlang-programming+unsubscribe@REDACTED > . > > For more options, visit this group athttp:// > groups.google.com/group/erlang-programming?hl=en. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gershon Bialer http://www.intervoo.com From bob@REDACTED Thu May 20 23:08:07 2010 From: bob@REDACTED (Bob Ippolito) Date: Thu, 20 May 2010 14:08:07 -0700 Subject: [erlang-questions] Re: Library management in erlang. In-Reply-To: References: Message-ID: It works something like this: $ cat rebar.config %% -*- erlang -*- {erl_opts, [%% fail_on_warning, debug_info]}. {deps, [{eldap, ".*", {git, "git://github.com/etrepum/eldap.git", ""}}, {seethrough, ".*", {git, "git://github.com/etrepum/seethrough_erl.git", ""}}]}. On Thu, May 20, 2010 at 1:07 PM, gershon bialer wrote: > Does rebar have a package management system based off github and mercurial > archives? I've heard something to that effect, but I have yet to see how > that actually works. > > On Thu, May 20, 2010 at 10:34 AM, Logan, Martin wrote: > >> We are working hard on it still. We have a secure website developed and are >> holding back on releasing it right now because of some other refactoring >> that is going on. ?Erlware should have most of the important code already >> stored for most versions of Erts. All the code comes with its src so you are >> free to rebuild it if you want. The caveat is that Erlware repos are not >> secure. They are quite open right now. The community is small enough such >> that we don't have problems. That said, when we finally push out our new >> site and repo that will change. >> >> It is also possible to host your own internal repo easily with a project >> called portius. ?Here is a link to a video explaining how to do it. >> http://www.youtube.com/watch?v=ogbz0Ram1_w >> >> Cheers, >> Martin >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On >> Behalf Of Steve Davis >> Sent: Monday, May 17, 2010 8:23 PM >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] Re: Library management in erlang. >> >> Closest I have seen are: >> >> http://projects.trapexit.org/web/ >> >> http://jungerl.sourceforge.net/ >> >> http://erlware.org/ >> >> /s >> >> On May 17, 7:35 pm, Tim Uckun wrote: >> > Is there anything equivalent to ruby gems in erlang? >> > >> > I am trying to follow a couple of tutorials they all say "go download >> > this library, git clone this library" etc. ?How do you guys keep your >> > libraries current and secure? is there an equivalent of CPAN or >> > rubyforge where I can search for libraries that might be of use? >> > >> > Are there the equivalent of ?gem search, gem list, gem install, gem >> > update,gem uninstall etc. >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > Seehttp://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> "Erlang Programming" group. >> > To post to this group, send email to erlang-programming@REDACTED >> . >> > To unsubscribe from this group, send email to >> erlang-programming+unsubscribe@REDACTED >> . >> > For more options, visit this group athttp:// >> groups.google.com/group/erlang-programming?hl=en. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- > Gershon Bialer > http://www.intervoo.com > From silent_vendetta@REDACTED Fri May 21 00:32:55 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Thu, 20 May 2010 15:32:55 -0700 Subject: Number of Fragmented Tables In-Reply-To: References: ,,,, Message-ID: How many fragments are too much? By that I mean is there a point, which is NOT dependent on amount of data but ONLY on number of fragments, where there is a performance hit? _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 From kyleqian@REDACTED Fri May 21 02:42:53 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Fri, 21 May 2010 08:42:53 +0800 Subject: [erlang-questions] About open_port and epipe error In-Reply-To: References: Message-ID: Yes, I am using werl.exe, but using erl.exe get the same result. by now, I do not do any i/o yet. All .exe run correctly, but only give me a epipe error after exit. So how can I fix this problem? Anyone do this under windows os? From kyleqian@REDACTED Fri May 21 02:57:09 2010 From: kyleqian@REDACTED (=?GB2312?B?x67P/sP3?=) Date: Fri, 21 May 2010 08:57:09 +0800 Subject: [erlang-questions] About open_port and epipe error In-Reply-To: References: Message-ID: Sorry, this error is due to my executable file type. I created a Win32 executable file before, now I create a console executable file and everything is ok. 2010/5/21 ??? > Yes, I am using werl.exe, but using erl.exe get the same result. > by now, I do not do any i/o yet. All .exe run correctly, but only give me a > epipe error after exit. > > So how can I fix this problem? Anyone do this under windows os? > From elliot.murphy@REDACTED Fri May 21 05:18:07 2010 From: elliot.murphy@REDACTED (Elliot Murphy) Date: Thu, 20 May 2010 23:18:07 -0400 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: The Ubuntu dev team disabled wx stuff in Erlang for Ubuntu 10.04 (we may have done this for 9.10 also) because we needed the erlang core on the LiveCD for CouchDB but could not support wx in Ubuntu "main", and did not have time to split the packages so that the wx related stuff could be in universe instead. I am working on fixing this for the next release of Ubuntu. In the meantime another option is installing Erlang from the erlang-dev PPA on Launchpad, which I maintain. That version is unchanged from debian, which means you get it built with debug info and with the wx bindings included. Sorry for the inconvenience, there were some tough choices to get Erlang onto the LiveCD, and while those choices were right for the liveCD they don't make things easy for Erlang developers working on Ubuntu. I hope to improve that. If there are any other quirks with the Erlang builds on Ubuntu, please let me know and I will try to help. -- Elliot Murphy On May 19, 2010, at 16:28, caio ariede wrote: > I know this. I think that they removed Wx support because it's > dependencies. > > I can run wx:module_info() on both systems (mac and linux), but when > I run > wx:new() I get the same error: > > =ERROR REPORT==== 11-May-2010::10:01:54 === > WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux- > gnu in: > /usr/lib/erlang/lib/wx-0.98.2/ > priv > ** exception error: {load_driver,"No driver found"} > in function wxe_server:start/0 > in call from wx:new/1 > > Caio Ariede > http://caioariede.com/ > > > On Wed, May 19, 2010 at 2:27 PM, Doug Edmunds (gmail) > wrote: > >> All official Ubuntu builds of erlang simply have wx disabled. If you >>> want to use it, you have to build your own version of erlang. >>> >>> Ralf >>> >> >> Who makes that decision, and why? >> >> == dae >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From caio.ariede@REDACTED Fri May 21 15:28:12 2010 From: caio.ariede@REDACTED (caio ariede) Date: Fri, 21 May 2010 10:28:12 -0300 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: Just got it to run on Ubuntu machine. First I removed the current installation, and installed all again: sudo add-apt-repository ppa:erlang-dev/ppa sudo apt-get install erlang libwxgtk2.8-dbg wx2.8-headers wx:new() ran like a charm :) Anyone knows how can I get the driver file for the MacOSX platform? Caio Ariede http://caioariede.com/ On Fri, May 21, 2010 at 12:18 AM, Elliot Murphy wrote: > > The Ubuntu dev team disabled wx stuff in Erlang for Ubuntu 10.04 (we may have done this for 9.10 also) because we needed the erlang core on the LiveCD for CouchDB but could not support wx in Ubuntu "main", and did not have time to split the packages so that the wx related stuff could be in universe instead. I am working on fixing this for the next release of Ubuntu. In the meantime another option is installing Erlang from the erlang-dev PPA on Launchpad, which I ?maintain. That version is unchanged from debian, which means you get it built with debug info and with the wx bindings included. > > Sorry for the inconvenience, there were some tough choices to get Erlang onto the LiveCD, and while those choices were right for the liveCD they don't make things easy for Erlang developers working on Ubuntu. I hope to improve that. If there are any other quirks with the Erlang builds on Ubuntu, please let me know and I will try to help. > > -- > Elliot Murphy > > On May 19, 2010, at 16:28, caio ariede wrote: > >> I know this. I think that they removed Wx support because it's dependencies. >> >> I can run wx:module_info() on both systems (mac and linux), but when I run >> wx:new() I get the same error: >> >> =ERROR REPORT==== 11-May-2010::10:01:54 === >> WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: >> /usr/lib/erlang/lib/wx-0.98.2/ >> priv >> ** exception error: {load_driver,"No driver found"} >> ? ?in function ?wxe_server:start/0 >> ? ?in call from wx:new/1 >> >> Caio Ariede >> http://caioariede.com/ >> >> >> On Wed, May 19, 2010 at 2:27 PM, Doug Edmunds (gmail) >> >>> wrote: >> >>> All official Ubuntu builds of erlang simply have wx disabled. If you >>>> >>>> want to use it, you have to build your own version of erlang. >>>> >>>> Ralf >>>> >>> >>> Who makes that decision, and why? >>> >>> == dae >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> From caio.ariede@REDACTED Fri May 21 15:29:42 2010 From: caio.ariede@REDACTED (caio ariede) Date: Fri, 21 May 2010 10:29:42 -0300 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: Ops, just forgot that I needed to run apt-get update between add-apt-repository and apt-get install. Caio Ariede http://caioariede.com/ On Fri, May 21, 2010 at 10:28 AM, caio ariede wrote: > Just got it to run on Ubuntu machine. > > First I removed the current installation, and installed all again: > > sudo add-apt-repository ppa:erlang-dev/ppa > sudo apt-get install erlang libwxgtk2.8-dbg wx2.8-headers > > wx:new() ran like a charm :) > > Anyone knows how can I get the driver file for the MacOSX platform? > > Caio Ariede > http://caioariede.com/ > > > On Fri, May 21, 2010 at 12:18 AM, Elliot Murphy wrote: >> >> The Ubuntu dev team disabled wx stuff in Erlang for Ubuntu 10.04 (we may have done this for 9.10 also) because we needed the erlang core on the LiveCD for CouchDB but could not support wx in Ubuntu "main", and did not have time to split the packages so that the wx related stuff could be in universe instead. I am working on fixing this for the next release of Ubuntu. In the meantime another option is installing Erlang from the erlang-dev PPA on Launchpad, which I ?maintain. That version is unchanged from debian, which means you get it built with debug info and with the wx bindings included. >> >> Sorry for the inconvenience, there were some tough choices to get Erlang onto the LiveCD, and while those choices were right for the liveCD they don't make things easy for Erlang developers working on Ubuntu. I hope to improve that. If there are any other quirks with the Erlang builds on Ubuntu, please let me know and I will try to help. >> >> -- >> Elliot Murphy >> >> On May 19, 2010, at 16:28, caio ariede wrote: >> >>> I know this. I think that they removed Wx support because it's dependencies. >>> >>> I can run wx:module_info() on both systems (mac and linux), but when I run >>> wx:new() I get the same error: >>> >>> =ERROR REPORT==== 11-May-2010::10:01:54 === >>> WX ERROR: Could not find suitable 'wxe_driver' for x86_64-pc-linux-gnu in: >>> /usr/lib/erlang/lib/wx-0.98.2/ >>> priv >>> ** exception error: {load_driver,"No driver found"} >>> ? ?in function ?wxe_server:start/0 >>> ? ?in call from wx:new/1 >>> >>> Caio Ariede >>> http://caioariede.com/ >>> >>> >>> On Wed, May 19, 2010 at 2:27 PM, Doug Edmunds (gmail) >>> >>>> wrote: >>> >>>> All official Ubuntu builds of erlang simply have wx disabled. If you >>>>> >>>>> want to use it, you have to build your own version of erlang. >>>>> >>>>> Ralf >>>>> >>>> >>>> Who makes that decision, and why? >>>> >>>> == dae >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> > From alessandro.sivieri@REDACTED Fri May 21 15:59:11 2010 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 21 May 2010 15:59:11 +0200 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: Unfortunately, the ppa team has the Karmic version but not the Lucid one... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From enewhuis@REDACTED Fri May 21 17:18:17 2010 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Fri, 21 May 2010 10:18:17 -0500 Subject: Is there a standard file name extension for raw a raw erlang term that can be consulted? Message-ID: <1826ADED-1679-407E-A482-39AEE9BB472F@gmail.com> *.eterm *.erlang *.consultable LOL Maybe it doesn't really matter because one really needs to know context. But then there sure are a lot of *.xml files out there. From mevans@REDACTED Fri May 21 19:08:34 2010 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 21 May 2010 13:08:34 -0400 Subject: Monitor a process vs a node Message-ID: Hi, We have a distributed Erlang system, and often run into the issue where we need to monitor processes on remote nodes. The normal way to do this is erlang:monitor(process,Pid). I have a few questions with this approach: 1) Are we guaranteed to get the {'DOWN', MonitorRef, Type, Object, Info} if the remote node crashes? 2) Is there a cost in doing many erlang:monitor calls (especially over remote nodes)? 3) Would a better option (WRT node crash) to maintain a table of node id's and interested pids on that node, and do erlang:monitor_node/2 instead? Thanks Matt From khaladkar.kiran@REDACTED Sat May 22 14:36:59 2010 From: khaladkar.kiran@REDACTED (kiran khaladkar) Date: Sat, 22 May 2010 18:06:59 +0530 Subject: Behavior for writing web applications Message-ID: Hi, When working on erlang for almost more than 2 years now, I felt using there could have been more generic behaviours than the standard one?s i.e. gen_server, gen_fsm, gen_event. I wanted a behaviour that is suited for writing web applications Following is the attempt to write such a behaviour %%%------------------------------------------------------------------------- %%% Author: Kiran Khaladkar %%% Description: Behavior for writing web applications. %%% Created: Sat 5th Dec 2009 %%%------------------------------------------------------------------------- -module(gen_app). -export([behavior_info/1, start/3, start/2]). -behavior(gen_server). -record(gen_app_state, {app_name :: atom(), %%stores app name cookie :: string(), %%cookie to be stored to refer to client app_state :: tuple() %%state maintained by the apps extending this behavior }). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). -export([get_info/1, get_cookie/1, set_cookie/2, handle_sync_request/2, handle_async_request/2]). behavior_info(callbacks) -> [{init,1}, {handle_request, 2}, {respond, 1},{terminate,1}]. %% Function: start/3 %% Description: starts the application by AppName with Params and Options. %% Options are gen_app process options %% Return: {ok, Pid} start(AppName, Param, Options) -> gen_server:start_link(?MODULE, {AppName,Param}, Options). %% Function: start/2 %% Description: Starts the application by AppName with params but without Options %% Return: {ok, Pid} start(AppName, Param) -> gen_server:start_link(?MODULE, {AppName, Param}, []). init({AppName, Param}) -> {ok, AppInitCookie, AppInitState} = AppName:init(Param), {ok, #gen_app_state{app_name = AppName, cookie = AppInitCookie, app_state = AppInitState}}. %% Function: get_info/1 %% Return: returns state of the gen_app get_info(AppId) -> gen_app:handle_sync_request(AppId, get_state). get_cookie(AppId) -> handle_sync_request(AppId, get_cookie). set_cookie(AppId, Cookie) -> handle_sync_request(AppId, {set_cookie, Cookie}). %% Funtion: handle_async_request(/2 %% Description: API to handle asynchronous calls that return immidiately without results. %% Rerurn: {noreply, State} handle_async_request(AppId, Reqest) -> gen_server:handle_cast(AppId, {app_req, Reqest}), {ok, processing}. %% Function: handle_sync_request/2 %% Description: API to handle synchronous calls that return the result of the %% request after completion of the processing of that request %% Return: {reply, Reply, NewState} after completion of the processing handle_sync_request(AppId, Request) -> gen_server:handle_call(AppId, {app_req, Request}). %% Funtion: handle_cast/2 %% Desciption: handle cast messages to gen_app process %% Return: {noreply, #state{}} handle_cast({app_req, Request}, #gen_app_state{app_name=AppName, app_state=AppState}=State) -> {ok, NewAppState, Response}=AppName:handle_request(Request, AppState), AppName:respond(Response), {noreply, State#gen_app_state{app_state=NewAppState}}; handle_cast(_Msg, _State) -> {noreplay, _State}. %% Funtion: handle_call/3 %% Desciption: handle cast messages to gen_app process %% Return: {noreply, #state{}} handle_call(get_state,_From, State) -> {reply, app_state, State}; handle_call({app_req, Req},_From, #gen_app_state{app_name = AppName, app_state = AppState} = State) -> {ok, NewAppState, Response}=AppName:handle_request(Req, AppState), AppName:respond(Response), {reply, ok, State#gen_app_state{app_state=NewAppState}}; handle_call(get_cookie, _From, State) -> {reply, State#gen_app_state.cookie, State}; handle_call({set_cookie, Cookie}, _From, State)-> {reply, ok, State#gen_app_state{cookie = Cookie}}; handle_call(_Msg,_From, _State) -> Reply = _Msg, {reply, Reply, _State}. handle_info(_Info, _State) -> {noreply, _State}. code_change(_OldVsn, State, _Extra) -> {ok, State}. terminate(Reason, #gen_app_state{app_name=AppName, app_state=AppState} = _State) -> AppName:terminate(Reason, AppState). Need your comments on this... regards, Kiran From mark.fine@REDACTED Sun May 23 20:58:04 2010 From: mark.fine@REDACTED (Mark Fine) Date: Sun, 23 May 2010 11:58:04 -0700 Subject: xmerl and $' Message-ID: The character $' is being returned in an XML document, and when I use xmerl to process the XML document, I get multiple values instead of a single value: problem() -> Xml = "\nThere's a problem", {Doc, _} = xmerl_scan:string(Xml), [Value || #xmlText{value = Value} <- xmerl_xpath:string("//Key/text()", Doc)]. Which returns ["There","'s a problem"] instead of ["There's a problem"]. I note if I pre-process the XML by s/&apos/'/, I will get the expected ["There's a problem"] -- is that the right workaround? Thanks! Mark From mark.fine@REDACTED Sun May 23 21:51:51 2010 From: mark.fine@REDACTED (Mark Fine) Date: Sun, 23 May 2010 12:51:51 -0700 Subject: xmerl and $' In-Reply-To: References: Message-ID: Ditto for $&, which I can't workaround: problem2() -> Xml = "\nThere & problem", {Doc, _} = xmerl_scan:string(Xml), [Value || #xmlText{value = Value} <- xmerl_xpath:string("//Key/text()", Doc)]. Which returns ["There ","& problem"] instead of ["There & problem"] I see in xmerl_scan:scan_entity_ref there's some handling of these and some comments that seem relevant: %% Chapter 4.4.2: ... the replacement text of entities used to escape %% markup delimiters (the entities amp, lt, gt, apos, quot) is always treated %% as data. (The string "AT&T;" expands to "AT&T;" and the remaining %% ampersand is not recognized as an entity-reference delimiter.)" let me know how I should pre-process the XML to enable xmerl_scan to handle things appropriately. Thanks! Mark On Sun, May 23, 2010 at 11:58 AM, Mark Fine wrote: > The character $' is being returned in an XML document, and when I use > xmerl to process the XML document, I get multiple values instead of a > single value: > > problem() -> > ? ?Xml = " encoding=\"UTF-8\"?>\nThere's a problem", > ? ?{Doc, _} = xmerl_scan:string(Xml), > ? ?[Value || #xmlText{value = Value} <- > xmerl_xpath:string("//Key/text()", Doc)]. > > Which returns ["There","'s a problem"] instead of ["There's a problem"]. > > I note if I pre-process the XML by s/&apos/'/, I will get the expected > ["There's a problem"] -- is that the right workaround? Thanks! > > Mark > From igorrs@REDACTED Sun May 23 22:46:55 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sun, 23 May 2010 17:46:55 -0300 Subject: Some tips on Mnesia In-Reply-To: References: Message-ID: Second and third posts: http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira wrote: > For those people who are starting to work with Mnesia, I'm writing a > short sequence of blog posts that are intended to point out some > relevant facts that don't seem to be well-known and could hurt you in > the beginning. > > I'll try to focus on the information that you can't google. ?;-) ?But > maybe in the end I'll also summarize the important tips that I learned > from other blogs. > > Here is the first post: > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html > > Good luck. > Igor. > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. From placements.narasimham@REDACTED Mon May 24 06:44:07 2010 From: placements.narasimham@REDACTED (narasimham placements) Date: Mon, 24 May 2010 10:14:07 +0530 Subject: [erlang-questions] Behavior for writing web applications In-Reply-To: References: Message-ID: Hi Kiran, Check Yaws documentation, you can find even better and simple way to handle web applications for your requirements. On Sat, May 22, 2010 at 6:06 PM, kiran khaladkar wrote: > Hi, > > When working on erlang for almost more than 2 years now, I felt using there > could have been more generic behaviours than the standard one?s i.e. > gen_server, gen_fsm, gen_event. I wanted a behaviour that is suited for > writing web applications > Following is the attempt to write such a behaviour > > > > %%%------------------------------------------------------------------------- > %%% Author: Kiran Khaladkar > %%% Description: Behavior for writing web applications. > %%% Created: Sat 5th Dec 2009 > > %%%------------------------------------------------------------------------- > > -module(gen_app). > -export([behavior_info/1, start/3, start/2]). > -behavior(gen_server). > > -record(gen_app_state, {app_name :: atom(), %%stores app name > cookie :: string(), %%cookie to be stored to refer to > client > app_state :: tuple() %%state maintained by the apps > extending this behavior > }). > > -export([init/1, handle_call/3, handle_cast/2, handle_info/2, > terminate/2, code_change/3]). > > -export([get_info/1, get_cookie/1, set_cookie/2, handle_sync_request/2, > handle_async_request/2]). > > behavior_info(callbacks) -> > [{init,1}, {handle_request, 2}, {respond, 1},{terminate,1}]. > > %% Function: start/3 > %% Description: starts the application by AppName with Params and Options. > %% Options are gen_app process options > %% Return: {ok, Pid} > start(AppName, Param, Options) -> > gen_server:start_link(?MODULE, {AppName,Param}, Options). > > %% Function: start/2 > %% Description: Starts the application by AppName with params but without > Options > %% Return: {ok, Pid} > start(AppName, Param) -> > gen_server:start_link(?MODULE, {AppName, Param}, []). > > init({AppName, Param}) -> > {ok, AppInitCookie, AppInitState} = AppName:init(Param), > {ok, #gen_app_state{app_name = AppName, cookie = AppInitCookie, > app_state = AppInitState}}. > > %% Function: get_info/1 > %% Return: returns state of the gen_app > get_info(AppId) -> > gen_app:handle_sync_request(AppId, get_state). > > get_cookie(AppId) -> > handle_sync_request(AppId, get_cookie). > > set_cookie(AppId, Cookie) -> > handle_sync_request(AppId, {set_cookie, Cookie}). > > %% Funtion: handle_async_request(/2 > %% Description: API to handle asynchronous calls that return immidiately > without results. > %% Rerurn: {noreply, State} > handle_async_request(AppId, Reqest) -> > gen_server:handle_cast(AppId, {app_req, Reqest}), > {ok, processing}. > > %% Function: handle_sync_request/2 > %% Description: API to handle synchronous calls that return the result of > the > %% request after completion of the processing of that request > %% Return: {reply, Reply, NewState} after completion of the processing > handle_sync_request(AppId, Request) -> > gen_server:handle_call(AppId, {app_req, Request}). > > %% Funtion: handle_cast/2 > %% Desciption: handle cast messages to gen_app process > %% Return: {noreply, #state{}} > handle_cast({app_req, Request}, #gen_app_state{app_name=AppName, > app_state=AppState}=State) -> > {ok, NewAppState, Response}=AppName:handle_request(Request, AppState), > AppName:respond(Response), > {noreply, State#gen_app_state{app_state=NewAppState}}; > > handle_cast(_Msg, _State) -> > {noreplay, _State}. > > %% Funtion: handle_call/3 > %% Desciption: handle cast messages to gen_app process > %% Return: {noreply, #state{}} > handle_call(get_state,_From, State) -> > {reply, app_state, State}; > > handle_call({app_req, Req},_From, #gen_app_state{app_name = AppName, > app_state = AppState} = State) -> > {ok, NewAppState, Response}=AppName:handle_request(Req, AppState), > AppName:respond(Response), > {reply, ok, State#gen_app_state{app_state=NewAppState}}; > > handle_call(get_cookie, _From, State) -> > {reply, State#gen_app_state.cookie, State}; > > handle_call({set_cookie, Cookie}, _From, State)-> > {reply, ok, State#gen_app_state{cookie = Cookie}}; > > handle_call(_Msg,_From, _State) -> > Reply = _Msg, > {reply, Reply, _State}. > > handle_info(_Info, _State) -> > {noreply, _State}. > > code_change(_OldVsn, State, _Extra) -> > {ok, State}. > > terminate(Reason, #gen_app_state{app_name=AppName, app_state=AppState} = > _State) -> > AppName:terminate(Reason, AppState). > > Need your comments on this... > regards, > Kiran > From rapsey@REDACTED Mon May 24 06:45:35 2010 From: rapsey@REDACTED (Rapsey) Date: Mon, 24 May 2010 06:45:35 +0200 Subject: [erlang-questions] Re: Some tips on Mnesia In-Reply-To: References: Message-ID: About that key value approach. My solution is a bit different. Let's say I have a database of nodes and store a list of IP addresses to them. I save them like so: {node,{address,NodeName}}, {IP,Port} This way I can query for that exact information mnesia:(dirty_)read, or I can get a list of all node addresses with mnesia:select - {node,{address,'_'}}, or get a list of all node information with {node,'_'}, or get all info about a node {node,{'_',NodeName}}. Sergej On Sun, May 23, 2010 at 10:46 PM, Igor Ribeiro Sucupira wrote: > Second and third posts: > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html > > On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira > wrote: > > For those people who are starting to work with Mnesia, I'm writing a > > short sequence of blog posts that are intended to point out some > > relevant facts that don't seem to be well-known and could hurt you in > > the beginning. > > > > I'll try to focus on the information that you can't google. ;-) But > > maybe in the end I'll also summarize the important tips that I learned > > from other blogs. > > > > Here is the first post: > > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html > > > > Good luck. > > Igor. > > > > -- > > "The secret of joy in work is contained in one word - excellence. To > > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From maruthavanan_s@REDACTED Mon May 24 08:11:15 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Mon, 24 May 2010 02:11:15 -0400 Subject: Performance on FIFO Message-ID: Hi, I need to implement a buffer that should hold a fixed length of records say for e.g. 20 with {key,Value} relationship, I expect the key would be integer and value to be list or binary upto length of 65536 so that I could transmit then in network.The older records should be removed and newer ones should be appended. I might lookup for Values based on keys. I could see many module where I can implement this like queue, dict, lists, proplists. Which one is most good for my situation? But I want my system to be fast performing, robust and should be able to Thanks, Marutha From mazen.harake@REDACTED Mon May 24 08:43:03 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 24 May 2010 09:43:03 +0300 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: Message-ID: <4BFA1FF7.5070906@erlang-solutions.com> And should be able to....? On 24/05/2010 09:11, maruthavanan s wrote: > Hi, > > I need to implement a buffer that should hold a fixed length of records say for e.g. 20 with {key,Value} relationship, I expect the key would be integer and value to be list or binary upto > length of 65536 so that I could transmit then in network.The older records should be removed and newer ones should be appended. I might lookup for Values based on keys. I could see many module where I can implement this like queue, dict, lists, proplists. > > Which one is most good for my situation? But I want my system to be fast performing, robust and should be able to > > Thanks, > Marutha > > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From jesper.louis.andersen@REDACTED Mon May 24 09:10:43 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 24 May 2010 09:10:43 +0200 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: Message-ID: On Mon, May 24, 2010 at 8:11 AM, maruthavanan s wrote: > I need to implement a buffer that should hold a fixed length of records say for e.g. 20 with {key,Value} relationship, I expect the key would be integer and value to be list or binary upto > length of 65536 so that I could transmit then in network.The older records should be removed and newer ones should be appended. I might lookup for Values based on keys. I could see many module where I can implement this like queue, dict, lists, proplists. Personal preference: I would go with queue first, lugging a count around for the size of the queue to keep it at 20. It will make you start quickly and since it has O(1) amortized access (if memory serves) you will have a fast implementation. I have some code which essentially does this for a network connection, and it is always the network connection which is the bottleneck in my case. And I only have 16K binary blocks in the queue. You have 1/4 the number of queue requests. You may want to optimize it later. Note that Erlang has a peculiar performance model. What you think is fast tend not to be and vice versa[*] [*] Haskell has an even more peculiar performance model, by the way :) -- J. From mail@REDACTED Mon May 24 09:16:53 2010 From: mail@REDACTED (Tim Fletcher) Date: Mon, 24 May 2010 00:16:53 -0700 (PDT) Subject: xmerl and $' In-Reply-To: References: Message-ID: <255c229c-78d7-4d09-9c92-2e897e1696e5@w3g2000vbd.googlegroups.com> > let me know how I should pre-process the XML to enable xmerl_scan > to handle things appropriately. Pre-processing the XML is unnecessary; just use lists:concat/1 or similar to combine the text values into a single string. Tim From Gustav.Simonsson.7871@REDACTED Mon May 24 09:37:39 2010 From: Gustav.Simonsson.7871@REDACTED (Gustav Simonsson) Date: Mon, 24 May 2010 09:37:39 +0200 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: Message-ID: <20100524093739.6t21v7s9wwwc8o4s@webmail.uu.se> Depends on what you think your bottleneck might be, what operations will be performed most often. Stdlib/queue might be a good start as you want a FIFO and are concerned about performance, it has O(1) for insertion and O(1) amortized, O(len(Q)) worst case for taking elements. BR, Gustav Simonsson Quoting maruthavanan s : > > Hi, > > I need to implement a buffer that should hold a fixed length of > records say for e.g. 20 with {key,Value} relationship, I expect the > key would be integer and value to be list or binary upto > length of 65536 so that I could transmit then in network.The older > records should be removed and newer ones should be appended. I might > lookup for Values based on keys. I could see many module where I > can implement this like queue, dict, lists, proplists. > > Which one is most good for my situation? But I want my system to be > fast performing, robust and should be able to > > Thanks, > Marutha > From eric.l.2046@REDACTED Mon May 24 09:42:36 2010 From: eric.l.2046@REDACTED (Eric Liang) Date: Mon, 24 May 2010 15:42:36 +0800 Subject: beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] Message-ID: <4BFA2DEC.50707@gmail.com> Hi all, I got the message( from /var/log/messages ) when I was benchmarking the ejabberd: May 24 14:55:26 dev-3 kernel: [264002.592197] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] While the ejabberd server works well, I am still curious about this message: what does it mean? and what can I do from this? However, any interesting information is appreciated, TIA. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From maruthavanan_s@REDACTED Mon May 24 09:56:45 2010 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Mon, 24 May 2010 03:56:45 -0400 Subject: [erlang-questions] Performance on FIFO In-Reply-To: <20100524093739.6t21v7s9wwwc8o4s@webmail.uu.se> References: ,<20100524093739.6t21v7s9wwwc8o4s@webmail.uu.se> Message-ID: Hi Again, Thanks for your valuable suggestions. I am sorry for incomplete mail. I want my system to be fast performing, robust and should be able to have good memory management. This data is run time and the queue size can increase up to 1000 My frequent operations would be insert and delete rather than lookup. Thanks, Marutha > Date: Mon, 24 May 2010 09:37:39 +0200 > From: Gustav.Simonsson.7871@REDACTED > To: maruthavanan_s@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Performance on FIFO > > Depends on what you think your bottleneck might be, what operations > will be performed most often. Stdlib/queue might be a good start as > you want a FIFO and are concerned about performance, it has O(1) for > insertion and O(1) amortized, O(len(Q)) worst case for taking elements. > > BR, > Gustav Simonsson > > Quoting maruthavanan s : > > > > > Hi, > > > > I need to implement a buffer that should hold a fixed length of > > records say for e.g. 20 with {key,Value} relationship, I expect the > > key would be integer and value to be list or binary upto > > length of 65536 so that I could transmit then in network.The older > > records should be removed and newer ones should be appended. I might > > lookup for Values based on keys. I could see many module where I > > can implement this like queue, dict, lists, proplists. > > > > Which one is most good for my situation? But I want my system to be > > fast performing, robust and should be able to > > > > Thanks, > > Marutha > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mikpe@REDACTED Mon May 24 11:17:17 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 24 May 2010 11:17:17 +0200 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <4BFA2DEC.50707@gmail.com> References: <4BFA2DEC.50707@gmail.com> Message-ID: <19450.17437.149260.11204@pilspetsen.it.uu.se> Eric Liang wrote: > Hi all, > I got the message( from /var/log/messages ) when I was benchmarking > the ejabberd: > > May 24 14:55:26 dev-3 kernel: [264002.592197] beam[8449]: segfault > at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in > beam[400000+174000] > > While the ejabberd server works well, I am still curious about this > message: what does it mean? and what can I do from this? However, any > interesting information is appreciated, TIA. It means an instance of the beam VM crashed with a NULL pointer dereference at instruction address 0x437e10. The large value for the stack pointer indicates that it's a 64-bit beam VM not a 32-bit one. It looks like a uni-processor version, otherwise the executable should have been `beam.smp'. If you want help to debug this, please give us some more information: 1. which version of Erlang/OTP is this? 2. what CPU type and OS was this executed on? 3. what tools (esp. gcc version) was this built with? 4. using gdb or objdump on the `beam' executable, can you provide a disassembly of the procedure surrounding 0x437e10? From per@REDACTED Mon May 24 14:04:08 2010 From: per@REDACTED (Per Hedeland) Date: Mon, 24 May 2010 14:04:08 +0200 (CEST) Subject: [erlang-questions] What is a tid()? In-Reply-To: <201004290853.o3T8rb9s092242@pluto.hedeland.org> Message-ID: <201005241204.o4OC4862014254@pluto.hedeland.org> Hi, I never saw any response to the below... If the fact that the ets module "claims" a builtin/"global" type tid() instead of ets:tid() is *not* a bug, is there perhaps some rationale for this? --Per Hedeland Per Hedeland wrote: > >Per Hedeland wrote: >> >>Kostis Sagonas wrote: >>> >>>Per Hedeland wrote: >>>> $ erlc test.erl >>>> ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name >>> >>>tid() is a built-in opaque type for ETS table identifiers. >> >>Thanks - maybe it's already in the queue for an update to >>http://www.erlang.org/doc/reference_manual/typespec.html? >> >>>These cannot be redefined. >>> >>>However, the warning you have run into predates the introduction of >>>remote types and is probably a bug. Although one should currently use >>>tid() to refer to ets:tid(), it should be possible to use a tid() type >>>that one of their modules export. >> >>Well, it can't be defined in the other module anyway, can it? If so >>that's perfectly OK by me, and the error is appropriate - it's just a >>documentation omission. > >Actually, I take that back - thinking about it, it seems very wrong that >this tid() should be built-in. Or rather, that it has the "global" name >tid() instead of only ets:tid(), which would be both clearer and more >logical (currently ets:tid() can't even be used). It's even listed along >with match_spec() (which *requires* the ets: prefix) in ets(3). > >As far as I can see (only looking in the documentation....), all the >other built-in / "global" types are defined in terms of the language, >either as fundamental types or derived from other types - whereas tid() >can only be defined in terms of a specific module (that the module >happens to be partially implemented in the VM shouldn't be relevant). > >Surely this is actually a bug, and the typespec documentation that omits >tid() is correct? > >--Per From davidw@REDACTED Mon May 24 14:28:47 2010 From: davidw@REDACTED (David Welton) Date: Mon, 24 May 2010 14:28:47 +0200 Subject: Applications, supervisors and supervisors, oh my Message-ID: Hi, For fun, I've been fooling around with this code: http://github.com/gdamjan/erlang-irc-bot To try and get back in the swing of Erlang. It's pretty good code, and the author is available on #erlang to talk about it. He's got some of the same questions as I do, so I thought I'd try and summarize what I'm wondering about. The API works like this: {ok, [{connection, Settings}]} = file:consult("settings.cfg"), {ok, IrcBot} = ircbot_server:start_link(Settings), gen_server:call(IrcBot, connect), gen_server:call(IrcBot, {add_plugin, plugins.twitter, []}), Which loads up some settings, starts the gen_server, connects it to the network, and then loads a plugin. So far so good. What I'm wondering is if this should be an "application". I want it to be restarted if it dies, so adding a supervisor makes sense. I'm not quite sure where applications fit in in terms of API's though... many that I've seen don't present much of an API, though: start/stop, and not a lot of ways of interacting with whatever's under the hood (or so it seems, perhaps I'm mistaken). The author of that code has added an application layer on top of it, and it seems to work that way too, which seems to limit the potential of the code if you want to include it in another application. I also want to be able to start up whatever I have created with -detached -s .... and not have to fiddle with it a bunch. I guess I'm looking for some thoughts/guidance on Erlang API design. I've looked through some of the online docs and the relevant bits of Francesco's book, but am still not entirely sure what the "best practices" in a case like this may be. Thank you, -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From kostis@REDACTED Mon May 24 14:33:43 2010 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 24 May 2010 15:33:43 +0300 Subject: [erlang-questions] What is a tid()? In-Reply-To: <201005241204.o4OC4862014254@pluto.hedeland.org> References: <201005241204.o4OC4862014254@pluto.hedeland.org> Message-ID: <4BFA7227.6000209@cs.ntua.gr> Per Hedeland wrote: > Hi, > > I never saw any response to the below... If the fact that the ets module > "claims" a builtin/"global" type tid() instead of ets:tid() is *not* a > bug, is there perhaps some rationale for this? I have made all changes that take out tid() from the list of built-in types. My intention is that this will be pushed to main OTP repository soon. Whether this change will be included in R14 or not is a matter of priorities of me and OTP developers and will also depend on the deadline for the code freeze. But this will eventually happen. Kostis > Per Hedeland wrote: >> Per Hedeland wrote: >>> Kostis Sagonas wrote: >>>> Per Hedeland wrote: >>>>> $ erlc test.erl >>>>> ./test.erl:5: referring to built-in type tid as a remote type; please take out the module name >>>> tid() is a built-in opaque type for ETS table identifiers. >>> Thanks - maybe it's already in the queue for an update to >>> http://www.erlang.org/doc/reference_manual/typespec.html? >>> >>>> These cannot be redefined. >>>> >>>> However, the warning you have run into predates the introduction of >>>> remote types and is probably a bug. Although one should currently use >>>> tid() to refer to ets:tid(), it should be possible to use a tid() type >>>> that one of their modules export. >>> Well, it can't be defined in the other module anyway, can it? If so >>> that's perfectly OK by me, and the error is appropriate - it's just a >>> documentation omission. >> Actually, I take that back - thinking about it, it seems very wrong that >> this tid() should be built-in. Or rather, that it has the "global" name >> tid() instead of only ets:tid(), which would be both clearer and more >> logical (currently ets:tid() can't even be used). It's even listed along >> with match_spec() (which *requires* the ets: prefix) in ets(3). >> >> As far as I can see (only looking in the documentation....), all the >> other built-in / "global" types are defined in terms of the language, >> either as fundamental types or derived from other types - whereas tid() >> can only be defined in terms of a specific module (that the module >> happens to be partially implemented in the VM shouldn't be relevant). >> >> Surely this is actually a bug, and the typespec documentation that omits >> tid() is correct? >> >> --Per > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From per@REDACTED Mon May 24 14:47:55 2010 From: per@REDACTED (Per Hedeland) Date: Mon, 24 May 2010 14:47:55 +0200 (CEST) Subject: [erlang-questions] What is a tid()? In-Reply-To: <4BFA7227.6000209@cs.ntua.gr> Message-ID: <201005241247.o4OClt6f015113@pluto.hedeland.org> Kostis Sagonas wrote: > >Per Hedeland wrote: >> Hi, >> >> I never saw any response to the below... If the fact that the ets module >> "claims" a builtin/"global" type tid() instead of ets:tid() is *not* a >> bug, is there perhaps some rationale for this? > >I have made all changes that take out tid() from the list of built-in >types. My intention is that this will be pushed to main OTP repository >soon. Whether this change will be included in R14 or not is a matter of >priorities of me and OTP developers and will also depend on the deadline >for the code freeze. But this will eventually happen. Great, thanks! --Per From mikpe@REDACTED Mon May 24 15:14:53 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Mon, 24 May 2010 15:14:53 +0200 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <4BFA71AE.5060708@gmail.com> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> Message-ID: <19450.31693.774522.332905@pilspetsen.it.uu.se> Liang Yupeng wrote: > Thanks for your reply, Mikael. Yes, it is beam.smp and 64-bit one. I have some doubts about that, see below. > > 3. what tools (esp. gcc version) was this built with? > > > I install erlang by the command apt-get: ... > Is this OK? Should I install a new beam-vm by source to get some debug info? Run `strings -a /path/to/beam | fgrep GCC | sort -u' (where /path/to/beam is the path to the beam executable). > > 4. using gdb or objdump on the `beam' executable, can you > > provide a disassembly of the procedure surrounding 0x437e10? > > sunny@REDACTED:~$ objdump -D /usr/lib/erlang/erts-5.7.2/bin/beam.smp > > beam.smp.objdump > sunny@REDACTED:~$ cat beam.smp.objdump | grep -C 10 437e1 > 437deb: 48 83 c4 38 add $0x38,%rsp > 437def: e9 8c fa ff ff jmpq 437880 > > 437df4: be 36 7e 55 00 mov $0x557e36,%esi > 437df9: 89 c7 mov %eax,%edi > 437dfb: e8 70 50 00 00 callq 43ce70 > > > 0000000000437e00 : > 437e00: 4c 89 6c 24 e8 mov %r13,-0x18(%rsp) > 437e05: 4c 8d ae 28 01 00 00 lea 0x128(%rsi),%r13 > 437e0c: 48 89 5c 24 d0 mov %rbx,-0x30(%rsp) > 437e11: 48 89 6c 24 d8 mov %rbp,-0x28(%rsp) > 437e16: 4c 89 64 24 e0 mov %r12,-0x20(%rsp) > 437e1b: 48 89 f3 mov %rsi,%rbx > 437e1e: 4c 89 74 24 f0 mov %r14,-0x10(%rsp) > 437e23: 4c 89 7c 24 f8 mov %r15,-0x8(%rsp) > 437e28: 41 89 fe mov %edi,%r14d > 437e2b: 48 83 ec 38 sub $0x38,%rsp > 437e2f: 4c 89 ef mov %r13,%rdi > 437e32: 48 89 d5 mov %rdx,%rbp > 437e35: 49 89 cc mov %rcx,%r12 > 437e38: e8 e3 f1 fe ff callq 427020 > > 437e3d: 85 c0 test %eax,%eax > 437e3f: 0f 85 ae 02 00 00 jne 4380f3 > > 437e45: 48 85 ed test %rbp,%rbp > > Is these lines enough? This makes me suspect even more that the crash is in an instance of the uni-processor beam executable, not the beam.smp executable. There are two signs for that: 1. The kernel message refers to the executable as `beam' not `beam.smp'. 2. The above disassembly from beam.smp doesn't HAVE any instruction starting at 0x437e10. So please do the objdump and grep thing again but on the plain `beam' executable. From james.hague@REDACTED Mon May 24 16:10:26 2010 From: james.hague@REDACTED (James Hague) Date: Mon, 24 May 2010 09:10:26 -0500 Subject: gb_trees:fold? Message-ID: There's gb_sets:fold, dict:fold, orddict:fold, but strangely no gb_trees:fold. Is that an intentional omission? I know about gb_trees:iterator, but note that there are both "iterator" and "fold" in gb_sets. From james.hague@REDACTED Mon May 24 17:26:50 2010 From: james.hague@REDACTED (James Hague) Date: Mon, 24 May 2010 10:26:50 -0500 Subject: [erlang-questions] gb_trees:fold? In-Reply-To: <4BFA96B7.1040304@gmail.com> References: <4BFA96B7.1040304@gmail.com> Message-ID: >It's an omission, but not intentional. Feel free to contribute. Good to know! It's trivial to write a version that uses "iterator" and "next," but that's not taking advantage of internal knowledge of gb_trees to avoid unnecessary allocations. Hmmm...gb_sets:fold looks simple enough to use as a model. From james.hague@REDACTED Mon May 24 18:03:05 2010 From: james.hague@REDACTED (James Hague) Date: Mon, 24 May 2010 11:03:05 -0500 Subject: [erlang-questions] gb_trees:fold? In-Reply-To: References: <4BFA96B7.1040304@gmail.com> Message-ID: Okay, here's gb_trees:fold/3: -spec fold(fun((term(), term(), term()) -> term()), term(), gb_tree()) -> term(). fold(F, A, {_, T}) when is_function(F, 3) -> fold_1(F, A, T). fold_1(F, Acc0, {Key, Value, Small, Big}) -> Acc1 = fold_1(F, Acc0, Small), Acc = F(Key, Value, Acc1), fold_1(F, Acc, Big); fold_1(_, Acc, _) -> Acc. Yes, I should submit this to git and add an entry to the docs, but that will have to wait for more time on my part. It was fun to discover how easy this was to write, regardless! From igorrs@REDACTED Mon May 24 20:49:55 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 24 May 2010 15:49:55 -0300 Subject: [erlang-questions] Re: Some tips on Mnesia In-Reply-To: References: Message-ID: On Mon, May 24, 2010 at 1:45 AM, Rapsey wrote: > About that key value approach. My solution is a bit different. Let's say I > have a database of nodes and store a list of IP addresses to them. I save > them like so: > {node,{address,NodeName}}, {IP,Port} I assume you meant {node, {address,NodeName}, {IP,Port}}, so that the key would be {address, NodeName}. Is that correct? The problem I was addressing was "adding a field in a flexible manner". With your solution, you would, for example, add a "status" field by inserting new keys like {status, NodeName}, with values that could be "active", "down" or whatever. Is that the idea? I'm just not sure you can efficiently retrieve all the information about a node. Won't the query for this key {'_',NodeName} perform a full table scan? Even if that query was very smart (or you knew in advance what fields you need), it would still need to retrieve several keys, which could even be in different nodes (in the case of a fragmented table). Certainly slower than reading one, longer record (as is the case with a "key-value" table). So, in short, compared to the "key-value" approach, your solution favors retrieving a single field (only slightly, if the records are smaller than a block/page), but turns it slower to retrieve all the fields of an entity. Best regards. Igor. > This way I can query for that exact information mnesia:(dirty_)read, or I > can get a list of all node addresses with mnesia:select - > {node,{address,'_'}}, or get a list of all node information with {node,'_'}, > or get all info about a node {node,{'_',NodeName}}. > > > Sergej > > On Sun, May 23, 2010 at 10:46 PM, Igor Ribeiro Sucupira wrote: > >> Second and third posts: >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html >> >> On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira >> wrote: >> > For those people who are starting to work with Mnesia, I'm writing a >> > short sequence of blog posts that are intended to point out some >> > relevant facts that don't seem to be well-known and could hurt you in >> > the beginning. >> > >> > I'll try to focus on the information that you can't google. ?;-) ?But >> > maybe in the end I'll also summarize the important tips that I learned >> > from other blogs. >> > >> > Here is the first post: >> > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html >> > >> > Good luck. >> > Igor. >> > >> > -- >> > "The secret of joy in work is contained in one word - excellence. To >> > know how to do something well is to enjoy it." - Pearl S. Buck. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From rapsey@REDACTED Mon May 24 21:26:07 2010 From: rapsey@REDACTED (Rapsey) Date: Mon, 24 May 2010 21:26:07 +0200 Subject: [erlang-questions] Re: Some tips on Mnesia In-Reply-To: References: Message-ID: On Mon, May 24, 2010 at 8:49 PM, Igor Ribeiro Sucupira wrote: > On Mon, May 24, 2010 at 1:45 AM, Rapsey wrote: > > About that key value approach. My solution is a bit different. Let's say > I > > have a database of nodes and store a list of IP addresses to them. I save > > them like so: > > {node,{address,NodeName}}, {IP,Port} > > I assume you meant {node, {address,NodeName}, {IP,Port}}, so that the > key would be {address, NodeName}. Is that correct? > > The problem I was addressing was "adding a field in a flexible > manner". With your solution, you would, for example, add a "status" > field by inserting new keys like {status, NodeName}, with values that > could be "active", "down" or whatever. Is that the idea? > > I'm just not sure you can efficiently retrieve all the information > about a node. Won't the query for this key {'_',NodeName} perform a > full table scan? > No it will not actually. I asked this question a on the mailing list not too long ago. As long as you don't use variable binding ('$1'), it will use the index on the first field. {node,{address,NodeName}} is the entire key, because I store more than just node information. This way I can use the same table for different kinds of data. > Even if that query was very smart (or you knew in advance what fields > you need), it would still need to retrieve several keys, which could > even be in different nodes (in the case of a fragmented table). > Certainly slower than reading one, longer record (as is the case with > a "key-value" table). > > Yes your solution might be better for fragmented tables. I don't really use mnesia in a distributed environment, so I used something that fits well with a single node. This way of saving data is a bit nicer, since you don't have to do a proplists:get_value after mnesia:read. But you can still query for more than one type of data at the same time. Sergej > So, in short, compared to the "key-value" approach, your solution > favors retrieving a single field (only slightly, if the records are > smaller than a block/page), but turns it slower to retrieve all the > fields of an entity. > > Best regards. > Igor. > > > This way I can query for that exact information mnesia:(dirty_)read, or I > > can get a list of all node addresses with mnesia:select - > > {node,{address,'_'}}, or get a list of all node information with > {node,'_'}, > > or get all info about a node {node,{'_',NodeName}}. > > > > > > Sergej > > > > On Sun, May 23, 2010 at 10:46 PM, Igor Ribeiro Sucupira < > igorrs@REDACTED>wrote: > > > >> Second and third posts: > >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html > >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html > >> > >> On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira > >> wrote: > >> > For those people who are starting to work with Mnesia, I'm writing a > >> > short sequence of blog posts that are intended to point out some > >> > relevant facts that don't seem to be well-known and could hurt you in > >> > the beginning. > >> > > >> > I'll try to focus on the information that you can't google. ;-) But > >> > maybe in the end I'll also summarize the important tips that I learned > >> > from other blogs. > >> > > >> > Here is the first post: > >> > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html > >> > > >> > Good luck. > >> > Igor. > >> > > >> > -- > >> > "The secret of joy in work is contained in one word - excellence. To > >> > know how to do something well is to enjoy it." - Pearl S. Buck. > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > From mark.fine@REDACTED Mon May 24 21:34:55 2010 From: mark.fine@REDACTED (Mark Fine) Date: Mon, 24 May 2010 12:34:55 -0700 Subject: [erlang-questions] Re: xmerl and $' In-Reply-To: <255c229c-78d7-4d09-9c92-2e897e1696e5@w3g2000vbd.googlegroups.com> References: <255c229c-78d7-4d09-9c92-2e897e1696e5@w3g2000vbd.googlegroups.com> Message-ID: Huh? In my actual application, I've got lots of values being returned in a single document -- I want to preserve these values, not concatenate them altogether. problem3() -> Xml = "\nThere's a problemThere&s another problem", {Doc, _} = xmerl_scan:string(Xml), [Value || #xmlText{value = Value} <- xmerl_xpath:string("//Key/text()", Doc)]. returns ["There","'s a problem","There","&s another problem"] instead of ["There's a problem","There&s another problem"]. And I definitely don't want ["There's a problemThere&s another problem"]. So far, preprocessing the XML seems necessary -- I go through and map ' to __APOS__ and & to __AMP__ before xmerl_scan:string(), and then map back -- this is the only way so far I've figured out how to make this work. Mark On Mon, May 24, 2010 at 12:16 AM, Tim Fletcher wrote: >> let me know how I should pre-process the XML to enable xmerl_scan >> to handle things appropriately. > > Pre-processing the XML is unnecessary; just use lists:concat/1 or > similar to combine the text values into a single string. > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mail@REDACTED Tue May 25 00:02:13 2010 From: mail@REDACTED (Tim Fletcher) Date: Mon, 24 May 2010 15:02:13 -0700 (PDT) Subject: xmerl and $' In-Reply-To: References: <255c229c-78d7-4d09-9c92-2e897e1696e5@w3g2000vbd.googlegroups.com> Message-ID: > Huh? In my actual application, I've got lots of values being > returned in a single document -- I want to preserve these values, > not concatenate them altogether. It looks like you want to extract the content of each 'Key' element as a single string, in which case you need to concatenate the values of the text/content nodes within each element. Like this: [lists:concat([Value || #xmlText{value=Value} <- Content]) || #xmlElement{content=Content} <- xmerl_xpath:string("//Key", Doc)]. Make sense? Tim From mark.fine@REDACTED Tue May 25 00:32:46 2010 From: mark.fine@REDACTED (Mark Fine) Date: Mon, 24 May 2010 15:32:46 -0700 Subject: [erlang-questions] Re: xmerl and $' In-Reply-To: References: <255c229c-78d7-4d09-9c92-2e897e1696e5@w3g2000vbd.googlegroups.com> Message-ID: Got it -- that works! Thanks! Mark On Mon, May 24, 2010 at 3:02 PM, Tim Fletcher wrote: >> Huh? In my actual application, I've got lots of values being >> returned in a single document -- I want to preserve these values, >> not concatenate them altogether. > > It looks like you want to extract the content of each 'Key' element as > a single string, in which case you need to concatenate the values of > the text/content nodes within each element. Like this: > > ?[lists:concat([Value || #xmlText{value=Value} <- Content]) || > #xmlElement{content=Content} <- xmerl_xpath:string("//Key", Doc)]. > > Make sense? > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bernie@REDACTED Tue May 25 01:03:04 2010 From: bernie@REDACTED (Bernard Duggan) Date: Tue, 25 May 2010 09:03:04 +1000 Subject: [erlang-questions] Applications, supervisors and supervisors, oh my In-Reply-To: References: Message-ID: <4BFB05A8.3010103@m5net.com> On 24/05/2010 10:28 PM, David Welton wrote: > What I'm wondering is if this should be an "application". I want it > to be restarted if it dies, so adding a supervisor makes sense. I'm > not quite sure where applications fit in in terms of API's though... > many that I've seen don't present much of an API, though: start/stop, > and not a lot of ways of interacting with whatever's under the hood > (or so it seems, perhaps I'm mistaken). > It being an 'application' does not inherently limit your interactions with it. Mnesia, for example, is an application and nobody would suggest you can't interact with it. For example, you can still talk to any registered processes within that application using their name. All (well, not all, but for the purposes of this discussion) an application provides is a way to start/stop a defined set of processes. If you want them to be supervised, the application in turn needs to start the worker processes within a supervisor - applications themselves don't provide any fault tolerance. Cheers, Bernard From dougedmunds@REDACTED Tue May 25 01:40:09 2010 From: dougedmunds@REDACTED (dae2010) Date: Mon, 24 May 2010 16:40:09 -0700 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> Message-ID: <4BFB0E59.3000501@douglasedmunds.com> I followed the instructions below for installing Erlang into Ubuntu. All the choices in the wx:demo() run, except one. If I select 'gl' from the list, it crashes, with this information in the shell: ** Reason for termination == ** {function_clause,[{wxSizer,add, [{wx_ref,64,wxStaticBoxSizer,[]}, {error,{no_gl_context,{gl,viewport,0}}}, [{proportion,1},{flag,8192}]]}, {demo,handle_event,2}, {wx_object,handle_msg,5}, {proc_lib,init_p_do_apply,3}]} =ERROR REPORT==== 24-May-2010::16:18:48 === WX INTERNAL ERROR: Failed to send return or event msg How do I interpret this? If something is not installed, what is the apt-get for it? The Ubuntu system: running Vbox within XP host, LinuxMint guest. -- Doug Edmunds On 5/21/2010 6:29 AM, caio ariede wrote: > Ops, just forgot that I needed to run apt-get update between > add-apt-repository and apt-get install. > > Caio Ariede > http://caioariede.com/ > > > > On Fri, May 21, 2010 at 10:28 AM, caio ariede wrote: >> Just got it to run on Ubuntu machine. >> >> First I removed the current installation, and installed all again: >> >> sudo add-apt-repository ppa:erlang-dev/ppa >> sudo apt-get install erlang libwxgtk2.8-dbg wx2.8-headers >> >> wx:new() ran like a charm :) >> From luismarianoguerra@REDACTED Tue May 25 01:46:05 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Mon, 24 May 2010 20:46:05 -0300 Subject: [ANN] efene 0.6 - a language for the erlang VM - released Message-ID: efene 0.6 Release Notes ----------------------- For the complete release notes with details on each item and examples see the release notes here: http://marianoguerra.com.ar/efene/docs/releases/osix.html About efene ~~~~~~~~~~~ efene is a programming language that runs on the erlang virtual machine. The idea is to provide an alternative syntax to erlang that is most suitable for people coming from languages like Java, C, C++, C#, Javascript. The language is almost 100% compatible with erlang (and will be), the compiler allows to translate an efene source file into a readable erlang one or compile it directly to bytecode. It also adds some syntactic sugar in some places to make some tasks easier. New Language Syntax ~~~~~~~~~~~~~~~~~~~ * Added support for global and local attributes * Objects implemented * Case expression allows match without parenthesis * Creation of records with all default values Language Syntax Changes ~~~~~~~~~~~~~~~~~~~~~~~ * Changes in record syntax to get an attribute * Public is not a keyword but a local attribute Compiler Changes ~~~~~~~~~~~~~~~~ * Improved error handling and printing * Show nice error message when a tab is found in ifene source * Output option added to the compiler: -t mod * Improved pretty printing * Output option -t erl2ast now works for complete modules * Allow to compile more than one file Internal Changes ~~~~~~~~~~~~~~~~ * Removed almost all the conflicts in the parser * Improved the project structure * Tests efene ast generation New Modules ~~~~~~~~~~~ A set of modules implemented, that provide functions to operate on objects, modules, perform validation, helpers, create REST APIs and object mapper for mongodb. Documentation ~~~~~~~~~~~~~ * Update website and readme * Documentation * Tutorial More Information ~~~~~~~~~~~~~~~~ * Download the latest snapshot: http://github.com/marianoguerra/efene/tarball/master * Website: http://marianoguerra.com.ar/efene * Documentation: http://marianoguerra.com.ar/efene/docs * Tutorial: http://marianoguerra.com.ar/efene/tutorial * Blog: http://efene.tumblr.com * Central repo: http://github.com/marianoguerra/efene (fork this one * Tracker: http://github.com/marianoguerra/efene/issues * Rosetta code page: http://rosettacode.org/wiki/Efene From igorrs@REDACTED Tue May 25 02:58:25 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 24 May 2010 21:58:25 -0300 Subject: [erlang-questions] Re: Some tips on Mnesia In-Reply-To: References: Message-ID: On Mon, May 24, 2010 at 4:26 PM, Rapsey wrote: > On Mon, May 24, 2010 at 8:49 PM, Igor Ribeiro Sucupira > wrote: >> >> On Mon, May 24, 2010 at 1:45 AM, Rapsey wrote: >> >> I'm just not sure you can efficiently retrieve all the information >> about a node. Won't the query for this key {'_',NodeName} perform a >> full table scan? > > No it will not actually. I asked this question a on the mailing list not too > long ago. In fact, I was not supposing it performs a full table scan on what I said after that. But now I'm curious: I ran the following test and it seemed to clearly be performing a full table scan. (node1@REDACTED)1> mnesia:create_schema([node()]). ok (node1@REDACTED)2> mnesia:start(). ok (node1@REDACTED)3> mnesia:create_table(test, [{attributes, [k, v]}, {disc_only_copies, [node()]}]). {atomic,ok} (node1@REDACTED)4> lists:foreach(fun(N) -> mnesia:dirty_write({test, {foo, N, N + 1}, N}) end, lists:seq(1, 4000000)). %% Took a VERY long time. Try 2 million. ;-) ok (node1@REDACTED)5> timer:tc(mnesia, dirty_select, [test, [{{test, {foo, '_', 3}, '_'}, [], [ok]}]]). {4006819,[ok]} (node1@REDACTED)6> timer:tc(mnesia, dirty_select, [test, [{{test, {foo, 2, 3}, '_'}, [], [ok]}]]). {287,[ok]} (node1@REDACTED)7> timer:tc(mnesia, dirty_select, [test, [{{test, {foo, '_', 3}, '_'}, [], [ok]}]]). %% Repetition. {4109629,[ok]} (node1@REDACTED)8> timer:tc(mnesia, dirty_select, [test, [{{test, {foo, 2, 3}, '_'}, [], [ok]}]]). %% Repetition. {235,[ok]} Repeating the procedure with a ram_copies, ordered_set table, I got 349730 microseconds for the matching with the partial key and 23 microseconds for the exact match that returns the same key. Changing the ram_copies table type to set, I got 288281 microseconds for the matching with the partial key and 28 microseconds for the exact match that returns the same key. So either someone misinformed you or the query must be performed on a very special way. Is it the second option? Best regards. Igor. > As long as you don't use variable binding ('$1'), it will use the > index on the first field. {node,{address,NodeName}} is the entire key, > because I store more than just node information. This way I can use the same > table for different kinds of data. > >> >> Even if that query was very smart (or you knew in advance what fields >> you need), it would still need to retrieve several keys, which could >> even be in different nodes (in the case of a fragmented table). >> Certainly slower than reading one, longer record (as is the case with >> a "key-value" table). >> > > Yes your solution might be better for fragmented tables. I don't really use > mnesia in a distributed environment, so I used something that fits well with > a single node. This way of saving data is a bit nicer, since you don't have > to do a proplists:get_value after mnesia:read. But you can still query for > more than one type of data at the same time. > > > > Sergej > > > > >> >> So, in short, compared to the "key-value" approach, your solution >> favors retrieving a single field (only slightly, if the records are >> smaller than a block/page), but turns it slower to retrieve all the >> fields of an entity. >> >> Best regards. >> Igor. >> >> > This way I can query for that exact information mnesia:(dirty_)read, or >> > I >> > can get a list of all node addresses with mnesia:select - >> > {node,{address,'_'}}, or get a list of all node information with >> > {node,'_'}, >> > or get all info about a node {node,{'_',NodeName}}. >> > >> > >> > Sergej >> > >> > On Sun, May 23, 2010 at 10:46 PM, Igor Ribeiro Sucupira >> > wrote: >> > >> >> Second and third posts: >> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html >> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html >> >> >> >> On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira >> >> wrote: >> >> > For those people who are starting to work with Mnesia, I'm writing a >> >> > short sequence of blog posts that are intended to point out some >> >> > relevant facts that don't seem to be well-known and could hurt you in >> >> > the beginning. >> >> > >> >> > I'll try to focus on the information that you can't google. ?;-) ?But >> >> > maybe in the end I'll also summarize the important tips that I >> >> > learned >> >> > from other blogs. >> >> > >> >> > Here is the first post: >> >> > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html >> >> > >> >> > Good luck. >> >> > Igor. >> >> > >> >> > -- >> >> > "The secret of joy in work is contained in one word - excellence. To >> >> > know how to do something well is to enjoy it." - Pearl S. Buck. >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> >> >> > >> >> >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From rvirding@REDACTED Tue May 25 03:09:07 2010 From: rvirding@REDACTED (Robert Virding) Date: Tue, 25 May 2010 03:09:07 +0200 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: <20100524093739.6t21v7s9wwwc8o4s@webmail.uu.se> Message-ID: I would myself probably specialise the queue module to have a fixed length buffer, which is what I understand you want. Sometimes it is easier to roll your own rather then to force something in the libraries into the wrong shaped whole. In this case their is not much code. Robert On 24 May 2010 09:56, maruthavanan s wrote: > > Hi Again, > > Thanks for your valuable suggestions. I am sorry for incomplete mail. > > I want my system to be fast performing, robust and > should be able to have good memory management. This data is run time and the queue size can increase up to 1000 > > My frequent operations would be insert and delete rather than lookup. > > Thanks, > Marutha > > >> Date: Mon, 24 May 2010 09:37:39 +0200 >> From: Gustav.Simonsson.7871@REDACTED >> To: maruthavanan_s@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Performance on FIFO >> >> Depends on what you think your bottleneck might be, what operations >> will be performed most often. Stdlib/queue might be a good start as >> you want a FIFO and are concerned about performance, it has O(1) for >> insertion and O(1) amortized, O(len(Q)) worst case for taking elements. >> >> BR, >> Gustav Simonsson >> >> Quoting maruthavanan s : >> >> > >> > Hi, >> > >> > I need to implement a buffer that should hold a fixed length of >> > records say for e.g. 20 with {key,Value} relationship, I expect the >> > key would be integer and value to be list or binary upto >> > length of 65536 so that I could transmit then in network.The older >> > records should be removed and newer ones should be appended. I might >> > ?lookup for Values based on keys. I could see many module where I >> > can ?implement this like queue, dict, lists, proplists. >> > >> > Which one is most good for my situation? But I want my system to be >> > fast performing, robust and should be able to >> > >> > Thanks, >> > Marutha >> > >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > From ok@REDACTED Tue May 25 04:12:44 2010 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 25 May 2010 14:12:44 +1200 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: ,<20100524093739.6t21v7s9wwwc8o4s@webmail.uu.se> Message-ID: On May 24, 2010, at 7:56 PM, maruthavanan s wrote: > > Hi Again, > > Thanks for your valuable suggestions. I am sorry for incomplete mail. > > I want my system to be fast performing, robust and > should be able to have good memory management. This data is run time > and the queue size can increase up to 1000 > > My frequent operations would be insert and delete rather than lookup. So you have a collection of {Key,Value} pairs, where from the collection's point of view it doesn't matter what the Values are, and all it knows is that Keys can be compared for equality. (This is a starting point, not a hard limitation.) The classic thing for fast (in the amortised sense) insert and delete is a pair of lists, where a b c d e f g A might be represented by [a,b,c] [g,f,e,d] L R where A = L ++ reverse(R). This way we can easily pull things off the front, and as easily put them on the back. This is exactly what the queue: module gives you. What it doesn't give you is lookup. With no "indexing" superstructure, it's going to be an O(N) linear scan, but if N is small (you mentioned 20) and lookups are rare, this is not a problem. %% lookup(Key, Queue) %% returns the leftmost {Key,Value} pair in Queue %% or 'none' if there is no such 2-tuple. lookup(Key, {Back,Front}) -> case lookup_front(Key, Front) of none -> lookup_back(Key, Back, none) ; Found -> Found end. lookup_front(Key, [Found={Key,_}|_]) -> Found; lookup_front(Key, [_|Front]) -> lookup_front(Key, Front); lookup_front(_, []) -> none. lookup_back(Key, [Found={Key,_}|Back], _) -> lookup_back(Key, Back, Found); lookup_back(Key, [_|Back], State) -> lookup_back(Key, Back, State); lookup_back(_, [], State) -> State. This has received some not very thorough testing. Make a copy of queue.erl and add this to it. Don't forget to -export lookup/2. Write the rest of your program. Run some test cases to find how how often the various operations really _are_ called, not how often you _think_ they are called. Then decide whether to replace your q module. I suggest From eric.l.2046@REDACTED Tue May 25 05:24:57 2010 From: eric.l.2046@REDACTED (Eric Liang) Date: Tue, 25 May 2010 11:24:57 +0800 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <19450.31693.774522.332905@pilspetsen.it.uu.se> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> <19450.31693.774522.332905@pilspetsen.it.uu.se> Message-ID: <4BFB4309.1060507@gmail.com> On 05/24/2010 09:14 PM, Mikael Pettersson wrote: > Liang Yupeng wrote: > >> Thanks for your reply, Mikael. Yes, it is beam.smp and 64-bit one. >> > I have some doubts about that, see below. > > >>> 3. what tools (esp. gcc version) was this built with? >>> >>> >> I install erlang by the command apt-get: >> > ... > >> Is this OK? Should I install a new beam-vm by source to get some debug info? >> > Run `strings -a /path/to/beam | fgrep GCC | sort -u' > (where /path/to/beam is the path to the beam executable). > sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam | fgrep GCC | sort -u sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam.smp | fgrep GCC | sort -u sunny@REDACTED:~$ You see, neither beam nor beam.smp contains the string like GCC. :( > >>> 4. using gdb or objdump on the `beam' executable, can you >>> provide a disassembly of the procedure surrounding 0x437e10? >>> >> sunny@REDACTED:~$ objdump -D /usr/lib/erlang/erts-5.7.2/bin/beam.smp > >> beam.smp.objdump >> sunny@REDACTED:~$ cat beam.smp.objdump | grep -C 10 437e1 >> 437deb: 48 83 c4 38 add $0x38,%rsp >> 437def: e9 8c fa ff ff jmpq 437880 >> >> 437df4: be 36 7e 55 00 mov $0x557e36,%esi >> 437df9: 89 c7 mov %eax,%edi >> 437dfb: e8 70 50 00 00 callq 43ce70 >> >> >> 0000000000437e00 : >> 437e00: 4c 89 6c 24 e8 mov %r13,-0x18(%rsp) >> 437e05: 4c 8d ae 28 01 00 00 lea 0x128(%rsi),%r13 >> 437e0c: 48 89 5c 24 d0 mov %rbx,-0x30(%rsp) >> 437e11: 48 89 6c 24 d8 mov %rbp,-0x28(%rsp) >> 437e16: 4c 89 64 24 e0 mov %r12,-0x20(%rsp) >> 437e1b: 48 89 f3 mov %rsi,%rbx >> 437e1e: 4c 89 74 24 f0 mov %r14,-0x10(%rsp) >> 437e23: 4c 89 7c 24 f8 mov %r15,-0x8(%rsp) >> 437e28: 41 89 fe mov %edi,%r14d >> 437e2b: 48 83 ec 38 sub $0x38,%rsp >> 437e2f: 4c 89 ef mov %r13,%rdi >> 437e32: 48 89 d5 mov %rdx,%rbp >> 437e35: 49 89 cc mov %rcx,%r12 >> 437e38: e8 e3 f1 fe ff callq 427020 >> >> 437e3d: 85 c0 test %eax,%eax >> 437e3f: 0f 85 ae 02 00 00 jne 4380f3 >> >> 437e45: 48 85 ed test %rbp,%rbp >> >> Is these lines enough? >> > This makes me suspect even more that the crash is in an instance of > the uni-processor beam executable, not the beam.smp executable. > There are two signs for that: > 1. The kernel message refers to the executable as `beam' not `beam.smp'. > You are right. After looking back the benchmark process, I found the process: sunny 4796 0.9 1.8 134320 76444 ? Sl 10:58 0:00 /usr/lib/erlang/erts-5.7.2/bin/beam -A 256 -P 250000 -- -root /usr/lib/erlang -progname erl -- -home /home/sunny -noshell -noinput -noshell -noinput -master tsung_controller@REDACTED -name os_mon@REDACTED -s slave slave_start tsung_controller@REDACTED slave_waiter_2 -rsh ssh -noshell -noinput -setcookie tsung -smp disable This is a tsung monitor process which was started by the tsung controller to get the information from dev-3. > 2. The above disassembly from beam.smp doesn't HAVE any instruction starting > at 0x437e10. > > So please do the objdump and grep thing again but on the plain `beam' executable. > sunny@REDACTED:~/commands$ objdump -D /usr/lib/erlang/erts-5.7.2/bin/beam > beam.objdump sunny@REDACTED:~/commands$ cat beam.objdump | grep -C 10 437e10 437ddd: e9 35 ff ff ff jmpq 437d17 437de2: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) 437de8: 48 2d 20 e2 00 00 sub $0xe220,%rax 437dee: 31 d2 xor %edx,%edx 437df0: 48 f7 b7 10 03 00 00 divq 0x310(%rdi) 437df7: 05 c0 00 00 00 add $0xc0,%eax 437dfc: e9 16 ff ff ff jmpq 437d17 437e01: 66 66 66 66 66 66 2e nopw %cs:0x0(%rax,%rax,1) 437e08: 0f 1f 84 00 00 00 00 437e0f: 00 437e10: 4c 8b 06 mov (%rsi),%r8 437e13: 49 83 e0 f8 and $0xfffffffffffffff8,%r8 437e17: 49 81 f8 1f 02 00 00 cmp $0x21f,%r8 437e1e: 77 40 ja 437e60 437e20: 49 8d 50 e0 lea -0x20(%r8),%rdx 437e24: 48 c1 ea 03 shr $0x3,%rdx 437e28: 4c 8b 4e 08 mov 0x8(%rsi),%r9 437e2c: 4d 85 c9 test %r9,%r9 437e2f: 74 4f je 437e80 437e31: 48 8b 46 10 mov 0x10(%rsi),%rax 437e35: 49 89 41 10 mov %rax,0x10(%r9) -- 437fae: e8 ed fb ff ff callq 437ba0 437fb3: 4d 85 e4 test %r12,%r12 437fb6: 48 89 c3 mov %rax,%rbx 437fb9: 74 10 je 437fcb 437fbb: 48 8b 00 mov (%rax),%rax 437fbe: 48 83 e0 f8 and $0xfffffffffffffff8,%rax 437fc2: 49 39 c7 cmp %rax,%r15 437fc5: 0f 86 95 00 00 00 jbe 438060 437fcb: 48 89 de mov %rbx,%rsi 437fce: 48 89 ef mov %rbp,%rdi 437fd1: e8 3a fe ff ff callq 437e10 437fd6: 48 89 d8 mov %rbx,%rax 437fd9: 48 8b 5c 24 08 mov 0x8(%rsp),%rbx 437fde: 48 8b 6c 24 10 mov 0x10(%rsp),%rbp 437fe3: 4c 8b 64 24 18 mov 0x18(%rsp),%r12 437fe8: 4c 8b 6c 24 20 mov 0x20(%rsp),%r13 437fed: 4c 8b 74 24 28 mov 0x28(%rsp),%r14 437ff2: 4c 8b 7c 24 30 mov 0x30(%rsp),%r15 437ff7: 48 83 c4 38 add $0x38,%rsp 437ffb: c3 retq 437ffc: 0f 1f 40 00 nopl 0x0(%rax) -- 438081: 74 54 je 4380d7 438083: 4d 85 e4 test %r12,%r12 438086: 74 0c je 438094 438088: 48 8b 10 mov (%rax),%rdx 43808b: 48 83 e2 f8 and $0xfffffffffffffff8,%rdx 43808f: 4c 39 fa cmp %r15,%rdx 438092: 73 cc jae 438060 438094: 48 89 c6 mov %rax,%rsi 438097: 48 89 ef mov %rbp,%rdi 43809a: 48 89 04 24 mov %rax,(%rsp) 43809e: e8 6d fd ff ff callq 437e10 4380a3: 48 8b 04 24 mov (%rsp),%rax 4380a7: e9 2d ff ff ff jmpq 437fd9 4380ac: 0f 1f 40 00 nopl 0x0(%rax) 4380b0: 4c 8d ae e0 dd ff ff lea -0x2220(%rsi),%r13 4380b7: 48 ba ab aa aa aa aa mov $0xaaaaaaaaaaaaaaab,%rdx 4380be: aa aa aa 4380c1: 4c 89 e8 mov %r13,%rax 4380c4: 48 f7 e2 mul %rdx 4380c7: 48 c1 ea 09 shr $0x9,%rdx 4380cb: 44 8d aa 80 00 00 00 lea 0x80(%rdx),%r13d -- 438302: 48 c7 83 b0 00 00 00 movq $0x1000,0xb0(%rbx) 438309: 00 10 00 00 43830d: 48 c7 83 b8 00 00 00 movq $0x18,0xb8(%rbx) 438314: 18 00 00 00 438318: 48 c7 43 28 2f 60 53 movq $0x53602f,0x28(%rbx) 43831f: 00 438320: 48 c7 83 e8 00 00 00 movq $0x437f40,0xe8(%rbx) 438327: 40 7f 43 00 43832b: 48 c7 83 f0 00 00 00 movq $0x437d00,0xf0(%rbx) 438332: 00 7d 43 00 438336: 48 c7 83 f8 00 00 00 movq $0x437e10,0xf8(%rbx) 43833d: 10 7e 43 00 438341: 48 c7 83 00 01 00 00 movq $0x438480,0x100(%rbx) 438348: 80 84 43 00 43834c: 48 c7 83 08 01 00 00 movq $0x0,0x108(%rbx) 438353: 00 00 00 00 438357: 48 c7 83 10 01 00 00 movq $0x438100,0x110(%rbx) 43835e: 00 81 43 00 438362: 48 c7 83 18 01 00 00 movq $0x438100,0x118(%rbx) 438369: 00 81 43 00 43836d: 48 c7 83 20 01 00 00 movq $0x4385c0,0x120(%rbx) Well, the beam does have the instructions starting at 0x437e10. By the way, would you mind tell me where to get the beam vm sources? (or erts sources if there is an address) Thanks, Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From eric.l.2046@REDACTED Tue May 25 05:40:36 2010 From: eric.l.2046@REDACTED (Eric Liang) Date: Tue, 25 May 2010 11:40:36 +0800 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <4BFB4309.1060507@gmail.com> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> <19450.31693.774522.332905@pilspetsen.it.uu.se> <4BFB4309.1060507@gmail.com> Message-ID: <4BFB46B4.10105@gmail.com> On 05/25/2010 11:24 AM, Eric Liang wrote: > On 05/24/2010 09:14 PM, Mikael Pettersson wrote: >> Liang Yupeng wrote: >> >>> Thanks for your reply, Mikael. Yes, it is beam.smp and 64-bit one. >>> >> I have some doubts about that, see below. >> >> >>>> 3. what tools (esp. gcc version) was this built with? >>>> >>>> >>> I install erlang by the command apt-get: >>> >> ... >> >>> Is this OK? Should I install a new beam-vm by source to get some debug info? >>> >> Run `strings -a /path/to/beam | fgrep GCC | sort -u' >> (where /path/to/beam is the path to the beam executable). >> > > sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam | > fgrep GCC | sort -u > sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam.smp > | fgrep GCC | sort -u > sunny@REDACTED:~$ > > You see, neither beam nor beam.smp contains the string like GCC. :( > >> >>>> 4. using gdb or objdump on the `beam' executable, can you >>>> provide a disassembly of the procedure surrounding 0x437e10? >>>> >>> sunny@REDACTED:~$ objdump -D /usr/lib/erlang/erts-5.7.2/bin/beam.smp > >>> beam.smp.objdump >>> sunny@REDACTED:~$ cat beam.smp.objdump | grep -C 10 437e1 >>> 437deb: 48 83 c4 38 add $0x38,%rsp >>> 437def: e9 8c fa ff ff jmpq 437880 >>> >>> 437df4: be 36 7e 55 00 mov $0x557e36,%esi >>> 437df9: 89 c7 mov %eax,%edi >>> 437dfb: e8 70 50 00 00 callq 43ce70 >>> >>> >>> 0000000000437e00 : >>> 437e00: 4c 89 6c 24 e8 mov %r13,-0x18(%rsp) >>> 437e05: 4c 8d ae 28 01 00 00 lea 0x128(%rsi),%r13 >>> 437e0c: 48 89 5c 24 d0 mov %rbx,-0x30(%rsp) >>> 437e11: 48 89 6c 24 d8 mov %rbp,-0x28(%rsp) >>> 437e16: 4c 89 64 24 e0 mov %r12,-0x20(%rsp) >>> 437e1b: 48 89 f3 mov %rsi,%rbx >>> 437e1e: 4c 89 74 24 f0 mov %r14,-0x10(%rsp) >>> 437e23: 4c 89 7c 24 f8 mov %r15,-0x8(%rsp) >>> 437e28: 41 89 fe mov %edi,%r14d >>> 437e2b: 48 83 ec 38 sub $0x38,%rsp >>> 437e2f: 4c 89 ef mov %r13,%rdi >>> 437e32: 48 89 d5 mov %rdx,%rbp >>> 437e35: 49 89 cc mov %rcx,%r12 >>> 437e38: e8 e3 f1 fe ff callq 427020 >>> >>> 437e3d: 85 c0 test %eax,%eax >>> 437e3f: 0f 85 ae 02 00 00 jne 4380f3 >>> >>> 437e45: 48 85 ed test %rbp,%rbp >>> >>> Is these lines enough? >>> >> This makes me suspect even more that the crash is in an instance of >> the uni-processor beam executable, not the beam.smp executable. >> There are two signs for that: >> 1. The kernel message refers to the executable as `beam' not `beam.smp'. >> > You are right. After looking back the benchmark process, I found the > process: > > sunny 4796 0.9 1.8 134320 76444 ? Sl 10:58 0:00 > /usr/lib/erlang/erts-5.7.2/bin/beam -A 256 -P 250000 -- -root > /usr/lib/erlang -progname erl -- -home /home/sunny -noshell > -noinput -noshell -noinput -master tsung_controller@REDACTED > -name os_mon@REDACTED -s slave slave_start > tsung_controller@REDACTED slave_waiter_2 -rsh ssh -noshell > -noinput -setcookie tsung -smp disable > > This is a tsung monitor process which was started by the tsung > controller to get the information from dev-3. > >> 2. The above disassembly from beam.smp doesn't HAVE any instruction starting >> at 0x437e10. >> >> So please do the objdump and grep thing again but on the plain `beam' executable. >> > > sunny@REDACTED:~/commands$ objdump -D > /usr/lib/erlang/erts-5.7.2/bin/beam > beam.objdump > sunny@REDACTED:~/commands$ cat beam.objdump | grep -C 10 437e10 > 437ddd: e9 35 ff ff ff jmpq 437d17 > > 437de2: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > 437de8: 48 2d 20 e2 00 00 sub $0xe220,%rax > 437dee: 31 d2 xor %edx,%edx > 437df0: 48 f7 b7 10 03 00 00 divq 0x310(%rdi) > 437df7: 05 c0 00 00 00 add $0xc0,%eax > 437dfc: e9 16 ff ff ff jmpq 437d17 > > 437e01: 66 66 66 66 66 66 2e nopw %cs:0x0(%rax,%rax,1) > 437e08: 0f 1f 84 00 00 00 00 > 437e0f: 00 > 437e10: 4c 8b 06 mov (%rsi),%r8 > 437e13: 49 83 e0 f8 and $0xfffffffffffffff8,%r8 > 437e17: 49 81 f8 1f 02 00 00 cmp $0x21f,%r8 > 437e1e: 77 40 ja 437e60 > > 437e20: 49 8d 50 e0 lea -0x20(%r8),%rdx > 437e24: 48 c1 ea 03 shr $0x3,%rdx > 437e28: 4c 8b 4e 08 mov 0x8(%rsi),%r9 > 437e2c: 4d 85 c9 test %r9,%r9 > 437e2f: 74 4f je 437e80 > > 437e31: 48 8b 46 10 mov 0x10(%rsi),%rax > 437e35: 49 89 41 10 mov %rax,0x10(%r9) > -- > 437fae: e8 ed fb ff ff callq 437ba0 > > 437fb3: 4d 85 e4 test %r12,%r12 > 437fb6: 48 89 c3 mov %rax,%rbx > 437fb9: 74 10 je 437fcb > > 437fbb: 48 8b 00 mov (%rax),%rax > 437fbe: 48 83 e0 f8 and $0xfffffffffffffff8,%rax > 437fc2: 49 39 c7 cmp %rax,%r15 > 437fc5: 0f 86 95 00 00 00 jbe 438060 > > 437fcb: 48 89 de mov %rbx,%rsi > 437fce: 48 89 ef mov %rbp,%rdi > 437fd1: e8 3a fe ff ff callq 437e10 > > 437fd6: 48 89 d8 mov %rbx,%rax > 437fd9: 48 8b 5c 24 08 mov 0x8(%rsp),%rbx > 437fde: 48 8b 6c 24 10 mov 0x10(%rsp),%rbp > 437fe3: 4c 8b 64 24 18 mov 0x18(%rsp),%r12 > 437fe8: 4c 8b 6c 24 20 mov 0x20(%rsp),%r13 > 437fed: 4c 8b 74 24 28 mov 0x28(%rsp),%r14 > 437ff2: 4c 8b 7c 24 30 mov 0x30(%rsp),%r15 > 437ff7: 48 83 c4 38 add $0x38,%rsp > 437ffb: c3 retq > 437ffc: 0f 1f 40 00 nopl 0x0(%rax) > -- > 438081: 74 54 je 4380d7 > > 438083: 4d 85 e4 test %r12,%r12 > 438086: 74 0c je 438094 > > 438088: 48 8b 10 mov (%rax),%rdx > 43808b: 48 83 e2 f8 and $0xfffffffffffffff8,%rdx > 43808f: 4c 39 fa cmp %r15,%rdx > 438092: 73 cc jae 438060 > > 438094: 48 89 c6 mov %rax,%rsi > 438097: 48 89 ef mov %rbp,%rdi > 43809a: 48 89 04 24 mov %rax,(%rsp) > 43809e: e8 6d fd ff ff callq 437e10 > > 4380a3: 48 8b 04 24 mov (%rsp),%rax > 4380a7: e9 2d ff ff ff jmpq 437fd9 > > 4380ac: 0f 1f 40 00 nopl 0x0(%rax) > 4380b0: 4c 8d ae e0 dd ff ff lea -0x2220(%rsi),%r13 > 4380b7: 48 ba ab aa aa aa aa mov $0xaaaaaaaaaaaaaaab,%rdx > 4380be: aa aa aa > 4380c1: 4c 89 e8 mov %r13,%rax > 4380c4: 48 f7 e2 mul %rdx > 4380c7: 48 c1 ea 09 shr $0x9,%rdx > 4380cb: 44 8d aa 80 00 00 00 lea 0x80(%rdx),%r13d > -- > 438302: 48 c7 83 b0 00 00 00 movq $0x1000,0xb0(%rbx) > 438309: 00 10 00 00 > 43830d: 48 c7 83 b8 00 00 00 movq $0x18,0xb8(%rbx) > 438314: 18 00 00 00 > 438318: 48 c7 43 28 2f 60 53 movq $0x53602f,0x28(%rbx) > 43831f: 00 > 438320: 48 c7 83 e8 00 00 00 movq $0x437f40,0xe8(%rbx) > 438327: 40 7f 43 00 > 43832b: 48 c7 83 f0 00 00 00 movq $0x437d00,0xf0(%rbx) > 438332: 00 7d 43 00 > 438336: 48 c7 83 f8 00 00 00 movq $0x437e10,0xf8(%rbx) > 43833d: 10 7e 43 00 > 438341: 48 c7 83 00 01 00 00 movq $0x438480,0x100(%rbx) > 438348: 80 84 43 00 > 43834c: 48 c7 83 08 01 00 00 movq $0x0,0x108(%rbx) > 438353: 00 00 00 00 > 438357: 48 c7 83 10 01 00 00 movq $0x438100,0x110(%rbx) > 43835e: 00 81 43 00 > 438362: 48 c7 83 18 01 00 00 movq $0x438100,0x118(%rbx) > 438369: 00 81 43 00 > 43836d: 48 c7 83 20 01 00 00 movq $0x4385c0,0x120(%rbx) > > Well, the beam does have the instructions starting at 0x437e10. > > By the way, would you mind tell me where to get the beam vm sources? > (or erts sources if there is an address) > Forget this, I found it in the OTP sources. :P Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From dgud@REDACTED Tue May 25 09:12:50 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 25 May 2010 09:12:50 +0200 Subject: [erlang-questions] Impossible to get wxErlang running on Ubuntu/MacOSX x64 platform In-Reply-To: <4BFB0E59.3000501@douglasedmunds.com> References: <87eih8jg7r.fsf@netsys-it.de> <4BF41F75.1000701@gmail.com> <4BFB0E59.3000501@douglasedmunds.com> Message-ID: The packages needed to build erlang (including wx) on ubuntu can be found here: http://wiki.github.com/erlang/otp/installation /Dan On Tue, May 25, 2010 at 1:40 AM, dae2010 wrote: > I followed the instructions below for installing Erlang into Ubuntu. > All the choices in the wx:demo() run, except one. ?If I select 'gl' from the > list, it crashes, with this information in the shell: > > ** Reason for termination == > ** {function_clause,[{wxSizer,add, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{wx_ref,64,wxStaticBoxSizer,[]}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {error,{no_gl_context,{gl,viewport,0}}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{proportion,1},{flag,8192}]]}, > ? ? ? ? ? ? ? ? ? ? {demo,handle_event,2}, > ? ? ? ? ? ? ? ? ? ? {wx_object,handle_msg,5}, > ? ? ? ? ? ? ? ? ? ? {proc_lib,init_p_do_apply,3}]} > > =ERROR REPORT==== 24-May-2010::16:18:48 === > WX INTERNAL ERROR: Failed to send return or event msg > > How do I interpret this? If something is not installed, what is the apt-get > for it? > > The Ubuntu system: running Vbox within XP host, LinuxMint guest. > > -- Doug Edmunds > > On 5/21/2010 6:29 AM, caio ariede wrote: >> >> Ops, just forgot that I needed to run apt-get update between >> add-apt-repository and apt-get install. >> >> Caio Ariede >> http://caioariede.com/ >> >> >> >> On Fri, May 21, 2010 at 10:28 AM, caio ariede >> ?wrote: >>> >>> Just got it to run on Ubuntu machine. >>> >>> First I removed the current installation, and installed all again: >>> >>> sudo add-apt-repository ppa:erlang-dev/ppa >>> sudo apt-get install erlang libwxgtk2.8-dbg wx2.8-headers >>> >>> wx:new() ran like a charm :) >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From raimo+erlang-questions@REDACTED Tue May 25 10:27:51 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 25 May 2010 10:27:51 +0200 Subject: [erlang-questions] Performance on FIFO In-Reply-To: References: Message-ID: <20100525082751.GA6097@erix.ericsson.se> On Tue, May 25, 2010 at 02:12:44PM +1200, Richard O'Keefe wrote: > > On May 24, 2010, at 7:56 PM, maruthavanan s wrote: > > > > >Hi Again, > > > >Thanks for your valuable suggestions. I am sorry for incomplete mail. > > > >I want my system to be fast performing, robust and > >should be able to have good memory management. This data is run time > >and the queue size can increase up to 1000 > > > >My frequent operations would be insert and delete rather than lookup. > > So you have a collection of {Key,Value} pairs, > where from the collection's point of view it doesn't > matter what the Values are, and all it knows is that > Keys can be compared for equality. (This is a starting > point, not a hard limitation.) > > The classic thing for fast (in the amortised sense) > insert and delete is a pair of lists, where > a b c d e f g A > might be represented by > [a,b,c] [g,f,e,d] L R > where A = L ++ reverse(R). This way we can easily > pull things off the front, and as easily put them on > the back. > > This is exactly what the queue: module gives you. > > What it doesn't give you is lookup. With no "indexing" > superstructure, it's going to be an O(N) linear scan, > but if N is small (you mentioned 20) and lookups are > rare, this is not a problem. > > %% lookup(Key, Queue) > %% returns the leftmost {Key,Value} pair in Queue > %% or 'none' if there is no such 2-tuple. And if you do not want to hack the queue module, you can sacrifice some performance and do it with a wrapper: lookup(Key, Q) -> case queue:peek(queue:filter( fun ({K,_}) when K =:= Key -> true; (_) -> false end, Q)) of empty -> none; {value,KV} -> KV end. The question for you to answer is how much that hurts for a queue lenght 1000 as you mentioned, and how often that happens. > > lookup(Key, {Back,Front}) -> > case lookup_front(Key, Front) > of none -> lookup_back(Key, Back, none) > ; Found -> Found > end. > > lookup_front(Key, [Found={Key,_}|_]) -> > Found; > lookup_front(Key, [_|Front]) -> > lookup_front(Key, Front); > lookup_front(_, []) -> > none. > > lookup_back(Key, [Found={Key,_}|Back], _) -> > lookup_back(Key, Back, Found); > lookup_back(Key, [_|Back], State) -> > lookup_back(Key, Back, State); > lookup_back(_, [], State) -> > State. > > This has received some not very thorough testing. > Make a copy of queue.erl and add this to it. > Don't forget to -export lookup/2. > > Write the rest of your program. > Run some test cases to find how how often the > various operations really _are_ called, not how > often you _think_ they are called. > Then decide whether to replace your q module. > > > I suggest > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From phmander@REDACTED Tue May 25 11:47:09 2010 From: phmander@REDACTED (Peter-Henry Mander) Date: Tue, 25 May 2010 10:47:09 +0100 Subject: Announce: T-Build updated Message-ID: Hi everyone, I have uploaded a revised T-Mobile Erlang build system onto Googlecode: http://code.google.com/p/erlang-t-build/ The rambling pile of scripts has been reduced to a single Makefile (158 lines when stripped of comments and redundant spaces). All that is required is GNUmake 3.81 (the latest), and BSD flavour 'sed' and 'tr' ('hostname', 'which' and 'date' are not essential, and can be removed from the Makefile). It works on Linux, Solaris10 and Mac OSX. Using --jobs is highly recommended. It is very, very fast. Makefile is the Other Concurrent Programming Language! There is an additional script to reload freshly compiled beams into Erlang runtimes with -sname matching the names of the releases. Please send me your comments, bug reports and feature requests. Pete. From mikpe@REDACTED Tue May 25 13:05:47 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 25 May 2010 13:05:47 +0200 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <4BFB4309.1060507@gmail.com> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> <19450.31693.774522.332905@pilspetsen.it.uu.se> <4BFB4309.1060507@gmail.com> Message-ID: <19451.44811.595873.338827@pilspetsen.it.uu.se> Eric Liang wrote: > On 05/24/2010 09:14 PM, Mikael Pettersson wrote: > > Liang Yupeng wrote: > > =20 > >> Thanks for your reply, Mikael. Yes, it is beam.smp and 64-bit one. > >> =20 > > I have some doubts about that, see below. > > > > =20 > >>> 3. what tools (esp. gcc version) was this built with? > >>> > >>> =20 > >> I install erlang by the command apt-get: > >> =20 > > ... > > =20 > >> Is this OK? Should I install a new beam-vm by source to get some debug= > info? > >> =20 > > Run `strings -a /path/to/beam | fgrep GCC | sort -u' > > (where /path/to/beam is the path to the beam executable). > > =20 > > sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam | > fgrep GCC | sort -u > sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam.smp | > fgrep GCC | sort -u > sunny@REDACTED:~$ > > You see, neither beam nor beam.smp contains the string like GCC. :( That means your distro removed useful metadata from your binaries. > sunny@REDACTED:~/commands$ objdump -D > /usr/lib/erlang/erts-5.7.2/bin/beam > beam.objdump > sunny@REDACTED:~/commands$ cat beam.objdump | grep -C 10 437e10 > 437ddd: e9 35 ff ff ff jmpq 437d17 > > 437de2: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > 437de8: 48 2d 20 e2 00 00 sub $0xe220,%rax > 437dee: 31 d2 xor %edx,%edx > 437df0: 48 f7 b7 10 03 00 00 divq 0x310(%rdi) > 437df7: 05 c0 00 00 00 add $0xc0,%eax > 437dfc: e9 16 ff ff ff jmpq 437d17 > > 437e01: 66 66 66 66 66 66 2e nopw %cs:0x0(%rax,%rax,1) > 437e08: 0f 1f 84 00 00 00 00 > 437e0f: 00 > 437e10: 4c 8b 06 mov (%rsi),%r8 > 437e13: 49 83 e0 f8 and $0xfffffffffffffff8,%r8 > 437e17: 49 81 f8 1f 02 00 00 cmp $0x21f,%r8 > 437e1e: 77 40 ja 437e60 > > 437e20: 49 8d 50 e0 lea -0x20(%r8),%rdx > 437e24: 48 c1 ea 03 shr $0x3,%rdx > 437e28: 4c 8b 4e 08 mov 0x8(%rsi),%r9 > 437e2c: 4d 85 c9 test %r9,%r9 > 437e2f: 74 4f je 437e80 > > 437e31: 48 8b 46 10 mov 0x10(%rsi),%rax > 437e35: 49 89 41 10 mov %rax,0x10(%r9) Again, this shows that your distro butchered the symbol table in your Erlang executable (beam). In reality erts_gfalc_init is only two instructions long and subsequent instructions belong to other named functions in erts/emulator/beam/erl_goodfit_alloc.c, but here they are incorrectly labeled as erts_gfalc_init+0x???. Anyway, I did a build from source with gcc-4.4.4 and matched the object code above to what I got, and it seems that unlink_free_block() in erl_goodfit_alloc.c was called with a NULL `block' parameter. This could be from the call at the end of get_free_block(), or via the ->unlink_free_block callback in erl_alloc_util.c. We really need a call stack dump at the point of the crash. You can get a stack dump from the crash by attaching gdb to the soon-to-crash beam process. Now instead of being terminated gdb will get control of the process and you should be able to print a stack trace with bt or where. (This does require that there's a sufficient time window from the start of the application to the crash.) From davidw@REDACTED Tue May 25 14:50:39 2010 From: davidw@REDACTED (David Welton) Date: Tue, 25 May 2010 14:50:39 +0200 Subject: [erlang-questions] Applications, supervisors, and servers, oh my Message-ID: On Tue, May 25, 2010 at 1:03 AM, Bernard Duggan wrote: > On 24/05/2010 10:28 PM, David Welton wrote: >> What I'm wondering is if this should be an "application". ?I want it >> to be restarted if it dies, so adding a supervisor makes sense. ?I'm >> not quite sure where applications fit in in terms of API's though... >> many that I've seen don't present much of an API, though: start/stop, >> and not a lot of ways of interacting with whatever's under the hood >> (or so it seems, perhaps I'm mistaken). > It being an 'application' does not inherently limit your interactions > with it. ?Mnesia, for example, is an application and nobody would > suggest you can't interact with it. ?For example, you can still talk to > any registered processes within that application using their name. ?All > (well, not all, but for the purposes of this discussion) an application > provides is a way to start/stop a defined set of processes. ?If you want > them to be supervised, the application in turn needs to start the worker > processes within a supervisor - applications themselves don't provide > any fault tolerance. Thanks, this helps some. I suppose registering the process names with a value the user supplies is the best way of giving them a way to continue interacting with the system once it's been started. In a similar vein, I'm wondering about the architecture of some other code I'm using. I found this on the net, and started adapting it: http://github.com/davidw/erlang_twitter/blob/master/src/twitter_stream.erl It basically just utilizes the http streaming API and then hands off received chunks to a user supplied M/F/A. However, I'm wondering about how to handle errors. The code, as is, has all it needs to deal with pretty much any error. "If it ain't broke, don't fix it", but I'm curious if this sort of thing might be written in a more OTP style, and if so, how. For instance, in addition to real errors such as 'the network went down', there are also errors like authorization problems. The first requires that it be restarted, the second really should just fail all the way up, so that whoever can go in and give it a correct user/pass. The current system seems more fine-grained than what a supervisor could provide, or am I wrong? Thanks, -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From rpettit@REDACTED Tue May 25 21:46:14 2010 From: rpettit@REDACTED (Rick Pettit) Date: Tue, 25 May 2010 14:46:14 -0500 Subject: pg2 still busted in R13B04 Message-ID: A while ago someone had posted that pg2 appeared to be broken as of R13B03. I am doing some testing with R13B04, and am finding the same to be true. Namely, with R13B04, it is possible for the same process to appear in a group many times (even if that process only added itself once). This leads to all sorts of problems, and makes it impossible to use pg2 with this release. Is this going to be addressed in the next release, or should people steer clear of pg2 in place of a better alternative...? -Rick From jesper.louis.andersen@REDACTED Tue May 25 22:22:02 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 25 May 2010 22:22:02 +0200 Subject: [erlang-questions] pg2 still busted in R13B04 In-Reply-To: References: Message-ID: On Tue, May 25, 2010 at 9:46 PM, Rick Pettit wrote: > A while ago someone had posted that pg2 appeared to be broken as of R13B03. > > I am doing some testing with R13B04, and am finding the same to be true. If you have a failing test case, then by all means make it available to the Erlang developers. That way, it is many times easier to fix eventual problems. -- J. From rpettit@REDACTED Tue May 25 22:39:44 2010 From: rpettit@REDACTED (Rick Pettit) Date: Tue, 25 May 2010 15:39:44 -0500 Subject: [erlang-questions] pg2 still busted in R13B04 In-Reply-To: References: Message-ID: On Tue, May 25, 2010 3:22 pm, Jesper Louis Andersen wrote: > On Tue, May 25, 2010 at 9:46 PM, Rick Pettit wrote: >> A while ago someone had posted that pg2 appeared to be broken as of >> R13B03. >> >> I am doing some testing with R13B04, and am finding the same to be true. > > If you have a failing test case, then by all means make it available > to the Erlang developers. That way, it is many times easier to fix > eventual problems. I'm working on it, but in the meantime the following bug report appears to cover the problem I'm having: http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/1838 -Rick > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mevans@REDACTED Tue May 25 22:52:21 2010 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 25 May 2010 16:52:21 -0400 Subject: [erlang-questions] pg2 still busted in R13B04 In-Reply-To: References: Message-ID: If you are feeling brave edit pg2.erl and change the ets:update_counter(pg2_table...) calls in join_group and leave_group as shown below: join_group: try _ = ets:update_counter(pg2_table, Member_Name_Pid, {2, +1, 1, 1}) leave_group: try ets:update_counter(pg2_table, Member_Name_Pid, {2, -1, 0, 0}) of It's not a permanent solution, but it works: i.e: join_group(Name, Pid) -> Ref_Pid = {ref, Pid}, try _ = ets:update_counter(pg2_table, Ref_Pid, {4, +1}) catch _:_ -> {RPid, Ref} = do_monitor(Pid), true = ets:insert(pg2_table, {Ref_Pid, RPid, Ref, 1}), true = ets:insert(pg2_table, {{ref, Ref}, Pid}) end, Member_Name_Pid = {member, Name, Pid}, try _ = ets:update_counter(pg2_table, Member_Name_Pid, {2, +1, 1, 1}) catch _:_ -> true = ets:insert(pg2_table, {Member_Name_Pid, 1}), _ = [ets:insert(pg2_table, {{local_member, Name, Pid}}) || node(Pid) =:= node()], true = ets:insert(pg2_table, {{pid, Pid, Name}}) end. leave_group(Name, Pid) -> Member_Name_Pid = {member, Name, Pid}, try ets:update_counter(pg2_table, Member_Name_Pid, {2, -1, 0, 0}) of N -> if N =:= 0 -> true = ets:delete(pg2_table, {pid, Pid, Name}), _ = [ets:delete(pg2_table, {local_member, Name, Pid}) || node(Pid) =:= node()], true = ets:delete(pg2_table, Member_Name_Pid); true -> ok end, Ref_Pid = {ref, Pid}, case ets:update_counter(pg2_table, Ref_Pid, {4, -1}) of 0 -> [{Ref_Pid,RPid,Ref,0}] = ets:lookup(pg2_table, Ref_Pid), true = ets:delete(pg2_table, {ref, Ref}), true = ets:delete(pg2_table, Ref_Pid), true = erlang:demonitor(Ref, [flush]), kill_monitor_proc(RPid, Pid); _ -> ok end catch _:_ -> ok end. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rick Pettit Sent: Tuesday, May 25, 2010 4:40 PM To: Jesper Louis Andersen Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] pg2 still busted in R13B04 On Tue, May 25, 2010 3:22 pm, Jesper Louis Andersen wrote: > On Tue, May 25, 2010 at 9:46 PM, Rick Pettit wrote: >> A while ago someone had posted that pg2 appeared to be broken as of >> R13B03. >> >> I am doing some testing with R13B04, and am finding the same to be true. > > If you have a failing test case, then by all means make it available > to the Erlang developers. That way, it is many times easier to fix > eventual problems. I'm working on it, but in the meantime the following bug report appears to cover the problem I'm having: http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/1838 -Rick > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bernie@REDACTED Wed May 26 01:35:04 2010 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 26 May 2010 09:35:04 +1000 Subject: [erlang-questions] Applications, supervisors, and servers, oh my In-Reply-To: References: Message-ID: <4BFC5EA8.9010001@m5net.com> On 25/05/10 22:50, David Welton wrote: > I suppose registering the process names with > a value the user supplies is the best way of giving them a way to > continue interacting with the system once it's been started. > That's certainly one option, though I've rarely seen it used (though that in itself is not an argument against doing it). More common is for a process to register with a name it knows, then provide interface functions to access it. So if, for example, I have a gen_server module that I start with a function like this: start_link(Args) -> gen_server:start_link({local, ?MODULE}, ?MODULE, Args, []). and I then want to, say, synchronously "add one to x", I'd provide that feature it in the same module as an exported function: add_one_to_x(X) -> gen_server:call(?MODULE, {add_one_to_x, X}). and as the related gen_server handle_call function: handle_call({add_one_to_x, X}, _, State) -> % Do something with X and state {reply, ok, NewState}; Hopefully that makes sense. Now the client only has to call add_one_to_x(X), Since you're limited to having one instance of any given application running on a given node, the fact that the application does its own process naming (rather than one being passed in) doesn't matter (and gives the client one less thing to keep track of). This is a pretty common pattern throughout the OTP code (mnesia etc). > However, I'm wondering about how to handle errors. The code, as is, > has all it needs to deal with pretty much any error. > [snip] The current system seems more fine-grained than what a supervisor could > provide, or am I wrong? > Error handling isn't something I'm really confident enough with yet to give you any firm advice - there's a bunch of other threads recently discussing this stuff that are probably worth a read. The short answer for this case, though, is that supervisors deal with restarting (or not restarting) processes or groups of processes. If you want error recovery to take place while maintaining the state of your process, then obviously that has to be done within the process itself. Cheers, Bernard From jeffm@REDACTED Wed May 26 05:06:15 2010 From: jeffm@REDACTED (Jeff) Date: Wed, 26 May 2010 13:06:15 +1000 Subject: A safe Erlang for third party users Message-ID: <4BFC9027.8090603@ghostgun.com> Is there a way to create a safe subset of Erlang which you could give an unknown person access to which is run on your own system? My thinking is something along the lines of: you have a web site which has aspects which are best controlled by a language, eg complex security for who can see what, etc. So rather than provide a list of check boxes, slider, and what not; You provide a file upload form and the user can upload a file. This is then compiled, and filtered for allowed and side effect free calls, etc before being allowed to execute on the web server's erlang vm. The filtering, I would image, would either be on the source code, on some partial compile of the file and/or on the compiled beam and would only permit expressly allowed function calls and parameters. Hope that makes sense. This is only a thought experiment and I don't have any plans to try such a thing, but thought it was an interesting idea. Jeff. From tony.arcieri@REDACTED Wed May 26 05:12:00 2010 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Tue, 25 May 2010 21:12:00 -0600 Subject: [erlang-questions] A safe Erlang for third party users In-Reply-To: <4BFC9027.8090603@ghostgun.com> References: <4BFC9027.8090603@ghostgun.com> Message-ID: On Tue, May 25, 2010 at 9:06 PM, Jeff wrote: > Is there a way to create a safe subset of Erlang which you could give an > unknown person access to which is run on your own system? My thinking is > something along the lines of: you have a web site which has aspects which > are best controlled by a language, eg complex security for who can see what, > etc. So rather than provide a list of check boxes, slider, and what not; You > provide a file upload form and the user can upload a file. This is then > compiled, and filtered for allowed and side effect free calls, etc before > being allowed to execute on the web server's erlang vm. The filtering, I > would image, would either be on the source code, on some partial compile of > the file and/or on the compiled beam and would only permit expressly allowed > function calls and parameters. > > Hope that makes sense. This is only a thought experiment and I don't have > any plans to try such a thing, but thought it was an interesting idea. Sounds like you want a capability model for Erlang? http://en.wikipedia.org/wiki/Object-capability_model -- Tony Arcieri Medioh! A Kudelski Brand From jeffm@REDACTED Wed May 26 05:41:04 2010 From: jeffm@REDACTED (Jeff) Date: Wed, 26 May 2010 13:41:04 +1000 Subject: [erlang-questions] A safe Erlang for third party users In-Reply-To: References: <4BFC9027.8090603@ghostgun.com> Message-ID: <4BFC9850.3040308@ghostgun.com> On 26/05/10 1:12 PM, Tony Arcieri wrote: > Sounds like you want a capability model for Erlang? > > http://en.wikipedia.org/wiki/Object-capability_model > I was think of this as a work around for the lack of such a system. Jeff. From silent_vendetta@REDACTED Wed May 26 07:09:10 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Tue, 25 May 2010 22:09:10 -0700 Subject: mnesia:transform_table Message-ID: So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? _________________________________________________________________ The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 From rpettit@REDACTED Wed May 26 07:03:57 2010 From: rpettit@REDACTED (Rick Pettit) Date: Wed, 26 May 2010 00:03:57 -0500 Subject: [erlang-questions] pg2 still busted in R13B04 In-Reply-To: References: Message-ID: On Tue, May 25, 2010 3:52 pm, Evans, Matthew wrote: > If you are feeling brave edit pg2.erl and change the > ets:update_counter(pg2_table...) calls in join_group and leave_group as > shown below: > > join_group: > try _ = ets:update_counter(pg2_table, Member_Name_Pid, {2, +1, 1, 1}) > > leave_group: > try ets:update_counter(pg2_table, Member_Name_Pid, {2, -1, 0, 0}) of > > It's not a permanent solution, but it works: Thanks, but for now I've decided instead to go with a different solution (namely to recompile pg2.erl from a previous release (R11B) which has worked reliably for years). The new (er, old) pg2 seems to be working reliably once again. Thanks for your help Matthew, -Rick > i.e: > > join_group(Name, Pid) -> > Ref_Pid = {ref, Pid}, > try _ = ets:update_counter(pg2_table, Ref_Pid, {4, +1}) > catch _:_ -> > {RPid, Ref} = do_monitor(Pid), > true = ets:insert(pg2_table, {Ref_Pid, RPid, Ref, 1}), > true = ets:insert(pg2_table, {{ref, Ref}, Pid}) > end, > Member_Name_Pid = {member, Name, Pid}, > try _ = ets:update_counter(pg2_table, Member_Name_Pid, {2, +1, 1, 1}) > catch _:_ -> > true = ets:insert(pg2_table, {Member_Name_Pid, 1}), > _ = [ets:insert(pg2_table, {{local_member, Name, Pid}}) || > node(Pid) =:= node()], > true = ets:insert(pg2_table, {{pid, Pid, Name}}) > end. > > leave_group(Name, Pid) -> > Member_Name_Pid = {member, Name, Pid}, > try ets:update_counter(pg2_table, Member_Name_Pid, {2, -1, 0, 0}) of > N -> > if > N =:= 0 -> > true = ets:delete(pg2_table, {pid, Pid, Name}), > _ = [ets:delete(pg2_table, {local_member, Name, Pid}) > || > node(Pid) =:= node()], > true = ets:delete(pg2_table, Member_Name_Pid); > true -> > ok > end, > Ref_Pid = {ref, Pid}, > case ets:update_counter(pg2_table, Ref_Pid, {4, -1}) of > 0 -> > [{Ref_Pid,RPid,Ref,0}] = ets:lookup(pg2_table, > Ref_Pid), > true = ets:delete(pg2_table, {ref, Ref}), > true = ets:delete(pg2_table, Ref_Pid), > true = erlang:demonitor(Ref, [flush]), > kill_monitor_proc(RPid, Pid); > _ -> > ok > end > catch _:_ -> > ok > end. > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Rick Pettit > Sent: Tuesday, May 25, 2010 4:40 PM > To: Jesper Louis Andersen > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] pg2 still busted in R13B04 > > On Tue, May 25, 2010 3:22 pm, Jesper Louis Andersen wrote: >> On Tue, May 25, 2010 at 9:46 PM, Rick Pettit >> wrote: >>> A while ago someone had posted that pg2 appeared to be broken as of >>> R13B03. >>> >>> I am doing some testing with R13B04, and am finding the same to be >>> true. >> >> If you have a failing test case, then by all means make it available >> to the Erlang developers. That way, it is many times easier to fix >> eventual problems. > > I'm working on it, but in the meantime the following bug report appears to > cover the problem I'm having: > > http://permalink.gmane.org/gmane.comp.lang.erlang.bugs/1838 > > -Rick > >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dangud@REDACTED Wed May 26 07:46:21 2010 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 26 May 2010 07:46:21 +0200 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: On Wed, May 26, 2010 at 7:09 AM, Chris Hicks wrote: > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? I think it is best to update all the code that handles that table, so both the new and old records can be handled correctly, then update the mnesia table. /Dan From fritchie@REDACTED Wed May 26 08:13:51 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 26 May 2010 01:13:51 -0500 Subject: 2010 ACM Erlang Workshop reminder: papers! Message-ID: <28015.1274854431@snookles.snookles.com> Hi, all. We're a bit less than three (3) weeks away from the submission deadline for the 2010 ACM Erlang Workshop. As someone who's aspiring to write a paper despite all of the distractions of The Day Job(tm) and Paying The Bills(tm), I understand the dilemma. All of us on the program committee would like to see perfect papers submitted the weeks ahead of deadline, but we know that's mostly impossible. All accepted submissions will go through review and resubmission. If yours isn't perfect, you'll have additional time and guidance to sharpen your paper to razor perfection. :-) The important dates are: * Submission deadline: Friday, June 11, 2010 * Author notification: Monday, June 28, 2010 * Final submission: Monday, August 2, 2010 * Workshop date: September 30, 2010, Baltimore, Maryland, USA For more details, see the call for papers at: http://www.erlang.org/workshop/2010/ -Scott Workshop Chair, 2010 ACM Erlang Workshop From igorrs@REDACTED Wed May 26 08:52:23 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 26 May 2010 03:52:23 -0300 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: If you are still modelling the tables, you might also want to consider some approach to avoid running mnesia:transform_table/3, since it needs to rewrite all the records. For example, as described here: http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value And discussed in this thread (from the third post on): http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 Good luck. Igor. On Wed, May 26, 2010 at 2:09 AM, Chris Hicks wrote: > > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? > _________________________________________________________________ > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From ulf.wiger@REDACTED Wed May 26 09:57:30 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 26 May 2010 09:57:30 +0200 Subject: [erlang-questions] A safe Erlang for third party users In-Reply-To: <4BFC9027.8090603@ghostgun.com> References: <4BFC9027.8090603@ghostgun.com> Message-ID: <4BFCD46A.8020305@erlang-solutions.com> On 05/26/2010 05:06 AM, Jeff wrote: > Is there a way to create a safe subset of Erlang which you could give an > unknown person access to which is run on your own system? My thinking is > something along the lines of: you have a web site which has aspects > which are best controlled by a language, eg complex security for who can > see what, etc. So rather than provide a list of check boxes, slider, and > what not; You provide a file upload form and the user can upload a file. > This is then compiled, and filtered for allowed and side effect free > calls, etc before being allowed to execute on the web server's erlang > vm. The filtering, I would image, would either be on the source code, on > some partial compile of the file and/or on the compiled beam and would > only permit expressly allowed function calls and parameters. I did some work on that with erlhive: http://erlhive.sf.net "ErlHive - Safe Erlang Reloaded!" (EUC 2006) http://erlang.mirror.su.se/euc/06/proceedings/1630Wiger.ppt "Micro-fiddling with Erlhive" (blog article) http://ulf.wiger.net/weblog/2009/03/10/micro-fiddling-with-erlhive/ I can safely say that it is very, very difficult to do this in a sufficiently transparent way. I tried to come up with a reasonable set of compromises, and put in quite some work to try to execute as much mainstream erlang code as possible, but the low uptake may suggest that this was not enough (to be clear, I did it because it was an intriguing problem, so I'm not complaining). Anyway, the idea of erlhive was to make it possible to have a web site running hosted erlang code, where applications could export a public API and at the same time hide private code and data. The thing that stopped my progress was that I tried to come up with a good code management regime for the hosted apps, but didn't manage to wrap it up neatly. Afterwards, I have thought that the best way to proceed would be to lower the ambition level and treat Erlang as a multi-user active database back-end, which is also the jist of the blog article referenced above. BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From johanmon@REDACTED Wed May 26 12:31:29 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 26 May 2010 12:31:29 +0200 Subject: generic replication bahaviour Message-ID: Hi, has anyone played around with a generic replication behaviour in the style of gen_server? I've benchmarked a prototype and it works ok if the read to write ratio is high. The idea is to, rather than provide a toolbox of group membership functionality, provide a very simple way of setting up a replicated server; hopefully gaining some performance advantages. The target is small scale replication. I've so far benchmarked it on a quad-core cpu and it has its advantages, distributed benchmarking is in the pipeline. http://web.it.kth.se/~johanmon/replicator.html One question is how the interface should look like i.e. initialization and call-back functions. Below is how to implement a replicated ets table. Does it make sense to have a gen_replicated module or will each application be so different that one model does not fit. Another question is what operations to provide and what consistency to provide. As it is right now I provide three functions: atomic multicast (abcast): performed in the same order by all replicas local call (call): performed by the local replica, does not change the state atomic call (acall): performed in total order with the multicast messages The very tricky issue on how to handle state transfer is below implemented simply by passing the whole state in a message but another solution could be to give the new replica a file name etc. It might be these decisions that make any generic replication scheme useless. Comments are welcome, Johan -module(rets). -export([start/1, replicate/2, init/1, init_replica/1, join/2, terminate/2]). -export([handle_abcast/2, handle_call/2, handle_acall/2]). start(Node) -> gen_replicated:start(Node, ?MODULE, [], []). replicate(Node, Replica) -> gen_replicated:replicate(Node, ?MODULE, Replica, please, []). init(_Args) -> {ok, ets:new(noname, [])}. init_replica(Copy) -> Table = ets:new(noname, []), ets:insert(Table, Copy), {ok, Table}. join(Request, Table) -> case Request of please -> {accepted, ets:tab2list(Table)}; _ -> {denied, "say please"} end. terminate(_Reason, Table) -> ets:delete(Table). handle_abcast({insert, Entry}, Table) -> ets:insert(Table, Entry), {noreply, Table}; handle_abcast(stop, _Table) -> {stop, normal}. handle_call({lookup, Key}, Table) -> {reply, ets:lookup(Table, Key)}. handle_acall({update_element, Key, PosValue}, Table) -> {reply, ets:update_element(Table, Key, PosValue), Table}. -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From ulf.wiger@REDACTED Wed May 26 13:32:35 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 26 May 2010 13:32:35 +0200 Subject: [erlang-questions] generic replication bahaviour In-Reply-To: References: Message-ID: <4BFD06D3.1060506@erlang-solutions.com> On 05/26/2010 12:31 PM, Johan Montelius wrote: > > Hi, > > has anyone played around with a generic replication behaviour in the > style of gen_server? One of the examples of gen_leader was gdict - a replicated version of dict. It hasn't carried over to the most recent github repos of gen_leader (e.g. http://github.com/kirindave/gen_leader_revival), but can be found in the original version: http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/gen_leader/src/ It was never really intended as a full-blown practical utility. OTOH, it has no known bugs. :) BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From serge@REDACTED Wed May 26 14:06:40 2010 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 26 May 2010 08:06:40 -0400 Subject: Passing a pid to a driver Message-ID: <4BFD0ED0.1050302@aleynikov.org> What's the right way of converting erlang_pid* decoded from external binary format using ei_decode_pid() to ErlDrvTermData form that can be used in a driver to send it a message using driver_send_term()? Serge From eric.l.2046@REDACTED Wed May 26 15:41:35 2010 From: eric.l.2046@REDACTED (Eric Liang) Date: Wed, 26 May 2010 21:41:35 +0800 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <19451.44811.595873.338827@pilspetsen.it.uu.se> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> <19450.31693.774522.332905@pilspetsen.it.uu.se> <4BFB4309.1060507@gmail.com> <19451.44811.595873.338827@pilspetsen.it.uu.se> Message-ID: <4BFD250F.6020202@gmail.com> On 05/25/2010 07:05 PM, Mikael Pettersson wrote: > Eric Liang wrote: > >> On 05/24/2010 09:14 PM, Mikael Pettersson wrote: >> >>> Liang Yupeng wrote: >>> =20 >>> >>>> Thanks for your reply, Mikael. Yes, it is beam.smp and 64-bit one. >>>> =20 >>>> >>> I have some doubts about that, see below. >>> >>> =20 >>> >>>>> 3. what tools (esp. gcc version) was this built with? >>>>> >>>>> =20 >>>>> >>>> I install erlang by the command apt-get: >>>> =20 >>>> >>> ... >>> =20 >>> >>>> Is this OK? Should I install a new beam-vm by source to get some debug= >>>> >> info? >> >>>> =20 >>>> >>> Run `strings -a /path/to/beam | fgrep GCC | sort -u' >>> (where /path/to/beam is the path to the beam executable). >>> =20 >>> >> sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam | >> fgrep GCC | sort -u >> sunny@REDACTED:~$ strings -a /usr/lib/erlang/erts-5.7.2/bin/beam.smp | >> fgrep GCC | sort -u >> sunny@REDACTED:~$ >> >> You see, neither beam nor beam.smp contains the string like GCC. :( >> > That means your distro removed useful metadata from your binaries. > > >> sunny@REDACTED:~/commands$ objdump -D >> /usr/lib/erlang/erts-5.7.2/bin/beam > beam.objdump >> sunny@REDACTED:~/commands$ cat beam.objdump | grep -C 10 437e10 >> 437ddd: e9 35 ff ff ff jmpq 437d17 >> >> 437de2: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) >> 437de8: 48 2d 20 e2 00 00 sub $0xe220,%rax >> 437dee: 31 d2 xor %edx,%edx >> 437df0: 48 f7 b7 10 03 00 00 divq 0x310(%rdi) >> 437df7: 05 c0 00 00 00 add $0xc0,%eax >> 437dfc: e9 16 ff ff ff jmpq 437d17 >> >> 437e01: 66 66 66 66 66 66 2e nopw %cs:0x0(%rax,%rax,1) >> 437e08: 0f 1f 84 00 00 00 00 >> 437e0f: 00 >> 437e10: 4c 8b 06 mov (%rsi),%r8 >> 437e13: 49 83 e0 f8 and $0xfffffffffffffff8,%r8 >> 437e17: 49 81 f8 1f 02 00 00 cmp $0x21f,%r8 >> 437e1e: 77 40 ja 437e60 >> >> 437e20: 49 8d 50 e0 lea -0x20(%r8),%rdx >> 437e24: 48 c1 ea 03 shr $0x3,%rdx >> 437e28: 4c 8b 4e 08 mov 0x8(%rsi),%r9 >> 437e2c: 4d 85 c9 test %r9,%r9 >> 437e2f: 74 4f je 437e80 >> >> 437e31: 48 8b 46 10 mov 0x10(%rsi),%rax >> 437e35: 49 89 41 10 mov %rax,0x10(%r9) >> > Again, this shows that your distro butchered the symbol table in > your Erlang executable (beam). In reality erts_gfalc_init is only > two instructions long and subsequent instructions belong to other named > functions in erts/emulator/beam/erl_goodfit_alloc.c, but here they > are incorrectly labeled as erts_gfalc_init+0x???. > > Anyway, I did a build from source with gcc-4.4.4 and matched the > object code above to what I got, and it seems that unlink_free_block() > in erl_goodfit_alloc.c was called with a NULL `block' parameter. > This could be from the call at the end of get_free_block(), or via > the ->unlink_free_block callback in erl_alloc_util.c. We really > need a call stack dump at the point of the crash. > Thanks for your work, Mikael. I've done a build of the source, but it just can't match the object. How do you make it? I use the command: apt-get source to get the source, so it does have the same version with the object. I'm now working on get the debug-info of that package in apt-get sources, but it also stalled. However, I've asked this in ubuntu's mailling-list, it should be worked out soon( I'll reply this mail then). > You can get a stack dump from the crash by attaching gdb to the > soon-to-crash beam process. Now instead of being terminated gdb will > get control of the process and you should be able to print a stack > trace with bt or where. (This does require that there's a sufficient > time window from the start of the application to the crash.) > I've make a core dump 4 seconds before it crash, as mentioned above, because don't get the right symbols, it just with some quesion-marks: Core was generated by `/usr/lib/erlang/erts-5.7.2/bin/beam'. #0 0x00007f0a28ecd5a9 in ?? () (gdb) whe #0 0x00007f0a28ecd5a9 in ?? () #1 0x0000000000000000 in ?? () (gdb) yes, I've not get the core when it crashed. Maybe my OS setting on core dump needs to be fixed, or maybe the beam which started by epmd does not generate core? Anyway, this also needs to be worked out. Look back the initial error: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] Would you mind tell me what's the meaning of -ip-/-sp-, and what does -error 4- means? Thanks in advance. Also, any other suggestions would be appreciated. Thanks, Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From egil@REDACTED Wed May 26 14:41:19 2010 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 26 May 2010 14:41:19 +0200 Subject: Speedy unsort:shuffle/1,2 ? Message-ID: <4BFD16EF.6060206@erix.ericsson.se> I while back there were some talk about an unsort or shuffle operation for lists. I too was in need of this since it very convenient in testing, so I wrote one. However, there are some constraints on this function. It needs to be generic enough, it needs to be fast enough and consume as little memory as possible. Three constraints that seldom spell success when combined. My suggestion is included in this mail, but my question is: does anyone have a faster, more generic and less memory consuming shuffle function? My second concern is entropy. Does my function randomize the list enough? This approach will split the list continuously until it cannot be divided any further, then randomly deciding how it should be reassembled. This should be fast but does it give enough entropy? It is ok for my needs, I do not need it to be perfect. Any comments? Regards, Bj?rn-Egil -------------- next part -------------- A non-text attachment was scrubbed... Name: unsort.erl Type: text/x-erlang Size: 2458 bytes Desc: not available URL: From johanmon@REDACTED Wed May 26 17:14:12 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 26 May 2010 17:14:12 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD16EF.6060206@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: On Wed, 26 May 2010 14:41:19 +0200, Bj?rn-Egil Dahlberg wrote: > My suggestion is included in this mail, but my question is: does anyone > have a faster, more generic and less memory consuming shuffle function? I haven't done any test but I would think the execution time will be very much depending on the number of random:uniform/1 calls that you do. As it is now you make a lot of calls but do not utilize the entropy. A don't think a call to random:uniform(2) is much cheaper than random:uniform(8098809898) nor do I know how well random:uniform is implemented. Here is an idea: generate a SHA-512 binary, use the binary bit by bit to divide the list into two lists, if you run out of bits, generate a new pattern. Return the remaining bit pattern and the two lists (this is similar to split in qsort). Recursively apply to the first list and then the second list and if we do it right we can avoid doing the last append (me think). Hmm, tempted to giv it a go but have to pick up the kids. Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From silent_vendetta@REDACTED Wed May 26 17:08:35 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Wed, 26 May 2010 08:08:35 -0700 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: , Message-ID: Yes I am still modeling tables and I remembered reading your blog, I'm just making sure I understand all of my options beforehand. I'm trying to decide between using a list with key-value pairs, a dict and a proplist. Are there any major disadvantages to using one of those over the other? > From: igorrs@REDACTED > Date: Wed, 26 May 2010 03:52:23 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] mnesia:transform_table > > If you are still modelling the tables, you might also want to consider > some approach to avoid running mnesia:transform_table/3, since it > needs to rewrite all the records. > > For example, as described here: > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value > > And discussed in this thread (from the third post on): > http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 > > Good luck. > Igor. > > On Wed, May 26, 2010 at 2:09 AM, Chris Hicks > wrote: > > > > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? > > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? > > _________________________________________________________________ > > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. > > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 From igorrs@REDACTED Wed May 26 17:48:21 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 26 May 2010 12:48:21 -0300 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: I thought of using a dict or a gb_trees to store the fields of each entry, but I gave up in seconds (;-)), because the only advantage (compared to proplists) would be the speed to retrieve a field (what is very fast, anyway), once you have read the record from the database (which is the slowest part). Considering that the performance difference to find a field is not significant if the number of fields is not big, using a gb_trees would be just a waste of space, for the huge majority of the cases. On Wed, May 26, 2010 at 12:08 PM, Chris Hicks wrote: > > Yes I am still modeling tables and I remembered reading your blog, I'm just making sure I understand all of my options beforehand. I'm trying to decide between using a list with key-value pairs, a dict and a proplist. Are there any major disadvantages to using one of those over the other? > >> From: igorrs@REDACTED >> Date: Wed, 26 May 2010 03:52:23 -0300 >> To: silent_vendetta@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] mnesia:transform_table >> >> If you are still modelling the tables, you might also want to consider >> some approach to avoid running mnesia:transform_table/3, since it >> needs to rewrite all the records. >> >> For example, as described here: >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value >> >> And discussed in this thread (from the third post on): >> http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 >> >> Good luck. >> Igor. >> >> On Wed, May 26, 2010 at 2:09 AM, Chris Hicks >> wrote: >> > >> > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? >> > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? >> > _________________________________________________________________ >> > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. >> > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 >> >> >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > _________________________________________________________________ > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From egil@REDACTED Wed May 26 18:09:50 2010 From: egil@REDACTED (=?UTF-8?B?QmrDtnJuLUVnaWwgRGFobGJlcmc=?=) Date: Wed, 26 May 2010 18:09:50 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: <4BFD47CE.1070601@erix.ericsson.se> On 2010-05-26 17:14, Johan Montelius wrote: > On Wed, 26 May 2010 14:41:19 +0200, Bj?rn-Egil Dahlberg > wrote: > >> My suggestion is included in this mail, but my question is: does anyone >> have a faster, more generic and less memory consuming shuffle function? > > I haven't done any test but I would think the execution time will be > very much depending on the number of random:uniform/1 calls that you do. > As it is now you make a lot of calls but do not utilize the entropy. A > don't think a call to random:uniform(2) is much cheaper than > random:uniform(8098809898) nor do I know how well random:uniform is > implemented. You are of course completely right. There is a lot of unnecessary calls to random:uniform/1. One can always redefine the constraints for the fun. Instead of returning a number, return a binary. The fun is there to provide the user with a alternative random-function to the shuffle function, mainly for creating pseudo-random shuffles. shuffle/1 uses random:uniform/1 as default. > Here is an idea: > > generate a SHA-512 binary, use the binary bit by bit to divide the list > into two lists, if you run out of bits, generate a new pattern. Return > the remaining bit pattern and the two lists (this is similar to split in > qsort). Recursively apply to the first list and then the second list and > if we do it right we can avoid doing the last append (me think). +1, I like this idea. Easy to hook in crypto:rand_bytes/1 also. // Bj?rn-Egil From silent_vendetta@REDACTED Wed May 26 18:15:17 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Wed, 26 May 2010 09:15:17 -0700 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: ,, Message-ID: What do you see as the advantage of Proplist over a Dict though? > From: igorrs@REDACTED > Date: Wed, 26 May 2010 12:48:21 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] mnesia:transform_table > > I thought of using a dict or a gb_trees to store the fields of each > entry, but I gave up in seconds (;-)), because the only advantage > (compared to proplists) would be the speed to retrieve a field (what > is very fast, anyway), once you have read the record from the database > (which is the slowest part). > Considering that the performance difference to find a field is not > significant if the number of fields is not big, using a gb_trees would > be just a waste of space, for the huge majority of the cases. > > On Wed, May 26, 2010 at 12:08 PM, Chris Hicks > wrote: > > > > Yes I am still modeling tables and I remembered reading your blog, I'm just making sure I understand all of my options beforehand. I'm trying to decide between using a list with key-value pairs, a dict and a proplist. Are there any major disadvantages to using one of those over the other? > > > >> From: igorrs@REDACTED > >> Date: Wed, 26 May 2010 03:52:23 -0300 > >> To: silent_vendetta@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: Re: [erlang-questions] mnesia:transform_table > >> > >> If you are still modelling the tables, you might also want to consider > >> some approach to avoid running mnesia:transform_table/3, since it > >> needs to rewrite all the records. > >> > >> For example, as described here: > >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value > >> > >> And discussed in this thread (from the third post on): > >> http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 > >> > >> Good luck. > >> Igor. > >> > >> On Wed, May 26, 2010 at 2:09 AM, Chris Hicks > >> wrote: > >> > > >> > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? > >> > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? > >> > _________________________________________________________________ > >> > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. > >> > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 > >> > >> > >> > >> -- > >> "The secret of joy in work is contained in one word - excellence. To > >> know how to do something well is to enjoy it." - Pearl S. Buck. > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > _________________________________________________________________ > > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. > > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ The New Busy is not the old busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3 From igorrs@REDACTED Wed May 26 18:42:46 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 26 May 2010 13:42:46 -0300 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: Since a dict consumes more space, the advantage of the proplist is the size. Basically, the dict will give you a performance gain that's irrelevant (unless you have many columns) and use more space. For example: 1> List = [{"firstname", "Igor"}, {"lastname", "Sucupira"}, {"gender", "male"}, {"birthdate", "02/15/1981"}]. [{"firstname","Igor"}, {"lastname","Sucupira"}, {"gender","male"}, {"birthdate","02/15/1981"}] 2> Dict= dict:from_list(List). {dict,4,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[["lastname",83,117,99,117,112,105,114,97], ["gender",109,97,108,101]], [], [["birthdate",48,50,47,49,53,47,49,57,56,49]], [],[],[],[],[],[],[],[],[], [["firstname",73,103,111,114]], [],[],[]}}} 3> GB = gb_trees:from_orddict(orddict:from_list(List)). {4, {"gender","male", {"firstname","Igor",{"birthdate","02/15/1981",nil,nil},nil}, {"lastname","Sucupira",nil,nil}}} 4> erlang:byte_size(term_to_binary(List)). 97 5> erlang:byte_size(term_to_binary(Dict)). 195 6> erlang:byte_size(term_to_binary(GB)). 125 8> timer:tc(lists, keysearch, ["firstname", 1, List]). {2,{value,{"firstname","Igor"}}} 9> timer:tc(gb_trees, lookup, ["firstname", GB]). {3,{value,"Igor"}} 10> timer:tc(dict, find, ["firstname", Dict]). {4,{ok,"Igor"}} 11> timer:tc(lists, keysearch, ["firstname", 1, List]). {3,{value,{"firstname","Igor"}}} 12> timer:tc(gb_trees, lookup, ["firstname", GB]). {3,{value,"Igor"}} 13> timer:tc(dict, find, ["firstname", Dict]). {4,{ok,"Igor"}} On Wed, May 26, 2010 at 1:15 PM, Chris Hicks wrote: > > What do you see as the advantage of Proplist over a Dict though? > >> From: igorrs@REDACTED >> Date: Wed, 26 May 2010 12:48:21 -0300 >> To: silent_vendetta@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] mnesia:transform_table >> >> I thought of using a dict or a gb_trees to store the fields of each >> entry, but I gave up in seconds (;-)), because the only advantage >> (compared to proplists) would be the speed to retrieve a field (what >> is very fast, anyway), once you have read the record from the database >> (which is the slowest part). >> Considering that the performance difference to find a field is not >> significant if the number of fields is not big, using a gb_trees would >> be just a waste of space, for the huge majority of the cases. >> >> On Wed, May 26, 2010 at 12:08 PM, Chris Hicks >> wrote: >> > >> > Yes I am still modeling tables and I remembered reading your blog, I'm just making sure I understand all of my options beforehand. I'm trying to decide between using a list with key-value pairs, a dict and a proplist. Are there any major disadvantages to using one of those over the other? >> > >> >> From: igorrs@REDACTED >> >> Date: Wed, 26 May 2010 03:52:23 -0300 >> >> To: silent_vendetta@REDACTED >> >> CC: erlang-questions@REDACTED >> >> Subject: Re: [erlang-questions] mnesia:transform_table >> >> >> >> If you are still modelling the tables, you might also want to consider >> >> some approach to avoid running mnesia:transform_table/3, since it >> >> needs to rewrite all the records. >> >> >> >> For example, as described here: >> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value >> >> >> >> And discussed in this thread (from the third post on): >> >> http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 >> >> >> >> Good luck. >> >> Igor. >> >> >> >> On Wed, May 26, 2010 at 2:09 AM, Chris Hicks >> >> wrote: >> >> > >> >> > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? >> >> > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? >> >> > _________________________________________________________________ >> >> > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. >> >> > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 >> >> >> >> >> >> >> >> -- >> >> "The secret of joy in work is contained in one word - excellence. To >> >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> > >> > _________________________________________________________________ >> > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. >> > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 >> >> >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > _________________________________________________________________ > The New Busy is not the old busy. Search, chat and e-mail from your inbox. > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3 -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From jesper.louis.andersen@REDACTED Wed May 26 18:58:15 2010 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 26 May 2010 18:58:15 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD16EF.6060206@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: 2010/5/26 Bj?rn-Egil Dahlberg : > I while back there were some talk about an unsort or shuffle operation for > lists. I too was in need of this since it very convenient in testing, so I > wrote one. Etorrent has one in its utils library as well: http://github.com/jlouis/etorrent/blob/master/src/etorrent_utils.erl#L101 It works by creating an analogy to shuffling a deck of cards and I am somewhat sure it will uniformly shuffle (though I have not proved that it is the case). The code is originally by Taylor Campbell if memory serves. I also think this is an excellent candidate for a standard library function. Getting it right is quite difficult. Getting it fast and right is even more difficult. I also note that the binary-generation trick ought to be usable on this one. In Etorrent it is rarely called and every place where it is used, the client would work even if you omit the call. Hence, it has not seen much testing. -- J. From silent_vendetta@REDACTED Wed May 26 18:56:06 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Wed, 26 May 2010 09:56:06 -0700 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: , ,, Message-ID: Well, seeing as how I don't think the number of columns will get out of hand for me that is certainly enough of a reason to use proplists instead. Thank you. > From: igorrs@REDACTED > Date: Wed, 26 May 2010 13:42:46 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] mnesia:transform_table > > Since a dict consumes more space, the advantage of the proplist is the > size. Basically, the dict will give you a performance gain that's > irrelevant (unless you have many columns) and use more space. > > For example: > > 1> List = [{"firstname", "Igor"}, {"lastname", "Sucupira"}, {"gender", > "male"}, {"birthdate", "02/15/1981"}]. > [{"firstname","Igor"}, > {"lastname","Sucupira"}, > {"gender","male"}, > {"birthdate","02/15/1981"}] > 2> Dict= dict:from_list(List). > {dict,4,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > {{[["lastname",83,117,99,117,112,105,114,97], > ["gender",109,97,108,101]], > [], > [["birthdate",48,50,47,49,53,47,49,57,56,49]], > [],[],[],[],[],[],[],[],[], > [["firstname",73,103,111,114]], > [],[],[]}}} > 3> GB = gb_trees:from_orddict(orddict:from_list(List)). > {4, > {"gender","male", > {"firstname","Igor",{"birthdate","02/15/1981",nil,nil},nil}, > {"lastname","Sucupira",nil,nil}}} > > 4> erlang:byte_size(term_to_binary(List)). > 97 > 5> erlang:byte_size(term_to_binary(Dict)). > 195 > 6> erlang:byte_size(term_to_binary(GB)). > 125 > > 8> timer:tc(lists, keysearch, ["firstname", 1, List]). > {2,{value,{"firstname","Igor"}}} > 9> timer:tc(gb_trees, lookup, ["firstname", GB]). > {3,{value,"Igor"}} > 10> timer:tc(dict, find, ["firstname", Dict]). > {4,{ok,"Igor"}} > 11> timer:tc(lists, keysearch, ["firstname", 1, List]). > {3,{value,{"firstname","Igor"}}} > 12> timer:tc(gb_trees, lookup, ["firstname", GB]). > {3,{value,"Igor"}} > 13> timer:tc(dict, find, ["firstname", Dict]). > {4,{ok,"Igor"}} > > > On Wed, May 26, 2010 at 1:15 PM, Chris Hicks > wrote: > > > > What do you see as the advantage of Proplist over a Dict though? > > > >> From: igorrs@REDACTED > >> Date: Wed, 26 May 2010 12:48:21 -0300 > >> To: silent_vendetta@REDACTED > >> CC: erlang-questions@REDACTED > >> Subject: Re: [erlang-questions] mnesia:transform_table > >> > >> I thought of using a dict or a gb_trees to store the fields of each > >> entry, but I gave up in seconds (;-)), because the only advantage > >> (compared to proplists) would be the speed to retrieve a field (what > >> is very fast, anyway), once you have read the record from the database > >> (which is the slowest part). > >> Considering that the performance difference to find a field is not > >> significant if the number of fields is not big, using a gb_trees would > >> be just a waste of space, for the huge majority of the cases. > >> > >> On Wed, May 26, 2010 at 12:08 PM, Chris Hicks > >> wrote: > >> > > >> > Yes I am still modeling tables and I remembered reading your blog, I'm just making sure I understand all of my options beforehand. I'm trying to decide between using a list with key-value pairs, a dict and a proplist. Are there any major disadvantages to using one of those over the other? > >> > > >> >> From: igorrs@REDACTED > >> >> Date: Wed, 26 May 2010 03:52:23 -0300 > >> >> To: silent_vendetta@REDACTED > >> >> CC: erlang-questions@REDACTED > >> >> Subject: Re: [erlang-questions] mnesia:transform_table > >> >> > >> >> If you are still modelling the tables, you might also want to consider > >> >> some approach to avoid running mnesia:transform_table/3, since it > >> >> needs to rewrite all the records. > >> >> > >> >> For example, as described here: > >> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html#key_value > >> >> > >> >> And discussed in this thread (from the third post on): > >> >> http://forum.trapexit.org/viewtopic.php?p=55510&sid=cc3aec96ba9387b61b5d192ccd1de1a6 > >> >> > >> >> Good luck. > >> >> Igor. > >> >> > >> >> On Wed, May 26, 2010 at 2:09 AM, Chris Hicks > >> >> wrote: > >> >> > > >> >> > So when updating a record definition the updated record, when reloaded, is not compatible with the old version of the record, as far as I understand it. Updating a Mnesia table to handle the new definition is straightforward enough but, my question revolves around the rest of the running system. Mainly I want to know what is the general (if there is one) process to updating a running system in this manner? > >> >> > Should you create a function which locks the whole table, reloads the code and updated all the definitions at the same time? What happens to the rest of the running system when you update a record definition while a process is in the middle of working with that old record? > >> >> > _________________________________________________________________ > >> >> > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. > >> >> > http://www.windowslive.com/campaign/thenewbusy?tile=multicalendar&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_5 > >> >> > >> >> > >> >> > >> >> -- > >> >> "The secret of joy in work is contained in one word - excellence. To > >> >> know how to do something well is to enjoy it." - Pearl S. Buck. > >> >> > >> >> ________________________________________________________________ > >> >> erlang-questions (at) erlang.org mailing list. > >> >> See http://www.erlang.org/faq.html > >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> >> > >> > > >> > _________________________________________________________________ > >> > Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. > >> > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 > >> > >> > >> > >> -- > >> "The secret of joy in work is contained in one word - excellence. To > >> know how to do something well is to enjoy it." - Pearl S. Buck. > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > _________________________________________________________________ > > The New Busy is not the old busy. Search, chat and e-mail from your inbox. > > http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3 > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 From james.aimonetti@REDACTED Wed May 26 19:22:55 2010 From: james.aimonetti@REDACTED (James Aimonetti) Date: Wed, 26 May 2010 13:22:55 -0400 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD47CE.1070601@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> <4BFD47CE.1070601@erix.ericsson.se> Message-ID: <4BFD58EF.2010304@gmail.com> I had a method in Javascript that was based on the Fisher-Yates shuffle that I tried converting to Erlang. It essentially zips the given list with a sequence 1-N, and uses a combination of lists:nth/2 and proplists:delete/2 to create the shuffled list. I've not looked into the entropy or bias as I've just thrown this together but the simple testing seems to be positive. %% unsort -module(unsort) -export([shuffle/1]). shuffle(L) -> Len = length(L), PropL = lists:zip(lists:seq(1, Len), L), shuffle(PropL, Len, []). shuffle([], _, L) -> lists:reverse(L); shuffle([{_,V}], _, L) -> lists:reverse([V | L]); shuffle([{_,V} | T]=Ps, Len, L) -> case swap_idx(Len) of 1 -> shuffle(T, Len-1, [V | L]); SwapIdx -> {K1, V1} = lists:nth(SwapIdx, Ps), shuffle(proplists:delete(K1, Ps), Len-1, [V1 | L]) end. swap_idx(Len) -> trunc(random:uniform() * Len) + 1. %% N length list %% Create X shuffled lists test(N, X) -> crypto:start(), L = lists:seq(1,N), Md5s = lists:map(fun(_) -> string:to_upper(lists:flatten([io_lib:format("~2.16.0b", [Digest]) || <> <= crypto:md5(shuffle:shuffle(L))])) end, lists:seq(1,X)), gb_trees:to_list( lists:foldl(fun(Hash, Tree) -> case gb_trees:lookup(Hash, Tree) of {value, Count} -> gb_trees:update(Hash, Count+1, Tree); _ -> gb_trees:insert(Hash, 1, Tree) end end, gb_trees:empty(), Md5s)). %% Sample output 1> unsort:test(2, 100000). [{"050D144172D916D0846F839E0412E929",50325}, {"0CB988D042A7F28DD5FE2B55B3F5AC7A",49675}] 2> unsort:test(3, 100000). [{"3BAC2889507C76C73913B5CACEB75B6D",16668}, {"5289DF737DF57326FCDD22597AFB1FAC",16674}, {"5D582C1341F4783E2FDE0F07D0AF9A06",16581}, {"9DE228839B2CCC8D08071BCADA88CA6D",16797}, {"A867F6E8EBD4E3A90C6084C9738B506B",16656}, {"AE0F84A71C987607257309BF36DFB41A",16624}] 3> unsort:test(4, 100000). [{"08D6C05A21512A79A1DFEB9D2A8F262F",4217}, {"0A27E477F545F21001C975A0DA5826CF",4244}, {"0AA9093A8C91D9BAF3DDB40613BF6052",4092}, {"0AFB83810CCDA4270CCBB36128D7258B",4152}, {"0E4717467434E3A8AF70C101789434D4",4202}, {"21C49141D9D5062E9C8A3FA8591507A7",4081}, {"26E5799C9C28C4F750DE5289644CCBCB",4190}, {"3044356A01283AC9BF0E83A69A76DAFA",4303}, {"32E2F90B64F81FEDB13FD92F554194EF",4069}, {"428C536DB5E72C45CF09CCD7A229D482",4185}, {"46DEB0BCE99B407029591EC8858F8D76",4185}, {"5DA92947151FE75785B296B4EE680265",4109}, {"6F3FF712FAE51C9E6A34E5649B1DF983",4058}, {"A9380E383BA79F5FCB90CF0F3A6F61C1",4164}, {"C73CABEB6558ABA030BBA9CA49DCDD75",3996}, {"C7E1C6D5AE3E1AC8B876F500F26BF37B",4132}, {"CBEED65578A453CDA5B7F9899F33F63F",4124}, {"DC7C46E4A1969FAF25ED7F53A0E7993B",4337}, {"DE5FE0700986F246278524D88F24DB10",4200}, {"DF2CEBAAE30811FFA20BA50D8E3EF365",4066}, {"E182CD2D19E8C3555B7D63454FC2D1BB",4218}, {"E64FEF4E93468D853B99662B25D37193",4240}, {"E78F3DE3AAE140076643E5D6F8E4C4B9",4204}, {"E94E7EE783C0AEE87D19D87F071FA2FD",4232}] 4> On 05/26/2010 12:09 PM, Bj?rn-Egil Dahlberg wrote: > On 2010-05-26 17:14, Johan Montelius wrote: >> On Wed, 26 May 2010 14:41:19 +0200, Bj?rn-Egil Dahlberg >> wrote: >> >>> My suggestion is included in this mail, but my question is: does anyone >>> have a faster, more generic and less memory consuming shuffle function? >> >> I haven't done any test but I would think the execution time will be >> very much depending on the number of random:uniform/1 calls that you do. >> As it is now you make a lot of calls but do not utilize the entropy. A >> don't think a call to random:uniform(2) is much cheaper than >> random:uniform(8098809898) nor do I know how well random:uniform is >> implemented. > > You are of course completely right. There is a lot of unnecessary > calls to random:uniform/1. One can always redefine the constraints for > the fun. Instead of returning a number, return a binary. The fun is > there to provide the user with a alternative random-function to the > shuffle function, mainly for creating pseudo-random shuffles. > shuffle/1 uses random:uniform/1 as default. > >> Here is an idea: >> >> generate a SHA-512 binary, use the binary bit by bit to divide the list >> into two lists, if you run out of bits, generate a new pattern. Return >> the remaining bit pattern and the two lists (this is similar to split in >> qsort). Recursively apply to the first list and then the second list and >> if we do it right we can avoid doing the last append (me think). > > +1, I like this idea. Easy to hook in crypto:rand_bytes/1 also. > > // Bj?rn-Egil > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- James Aimonetti mobile: 314.809.6307 work: 540.459.2220 email: james.aimonetti@REDACTED website: http://jamesaimonetti.com From egil@REDACTED Wed May 26 19:40:39 2010 From: egil@REDACTED (=?UTF-8?B?QmrDtnJuLUVnaWwgRGFobGJlcmc=?=) Date: Wed, 26 May 2010 19:40:39 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: <4BFD5D17.6070207@erix.ericsson.se> On 2010-05-26 18:58, Jesper Louis Andersen wrote: > 2010/5/26 Bj?rn-Egil Dahlberg: >> I while back there were some talk about an unsort or shuffle operation for >> lists. I too was in need of this since it very convenient in testing, so I >> wrote one. > > Etorrent has one in its utils library as well: > > http://github.com/jlouis/etorrent/blob/master/src/etorrent_utils.erl#L101 > > It works by creating an analogy to shuffling a deck of cards and I am > somewhat sure it will uniformly shuffle (though I have not proved that > it is the case). The code is originally by Taylor Campbell if memory > serves. I also think this is an excellent candidate for a standard > library function. Getting it right is quite difficult. Getting it fast > and right is even more difficult. I also note that the > binary-generation trick ought to be usable on this one. I like it. Etorrents version is almost like my suggestion, same idea anyway. However, the code i submitted is ~10+ times faster. // Bj?rn-Egil From serge@REDACTED Wed May 26 19:58:17 2010 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 26 May 2010 13:58:17 -0400 Subject: [ANN] Erlang ZeroMQ bindings Message-ID: <4BFD6139.7050301@aleynikov.org> A beta 0.1 release of Erlang ZeroMQ bindings is available here: http://github.com/saleyn/erlzmq Documentation: http://saleyn.github.com/erlzmq/ Relevant links: ZeroMQ: http://zeromq.org Regards, Serge From mikpe@REDACTED Wed May 26 20:14:23 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 26 May 2010 20:14:23 +0200 Subject: [erlang-questions] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] In-Reply-To: <4BFD250F.6020202@gmail.com> References: <4BFA2DEC.50707@gmail.com> <19450.17437.149260.11204@pilspetsen.it.uu.se> <4BFA71AE.5060708@gmail.com> <19450.31693.774522.332905@pilspetsen.it.uu.se> <4BFB4309.1060507@gmail.com> <19451.44811.595873.338827@pilspetsen.it.uu.se> <4BFD250F.6020202@gmail.com> Message-ID: <19453.25855.346911.194353@pilspetsen.it.uu.se> Eric Liang wrote: > I've done a build of the source, but it just can't match the object. How > do you make it? I use the command: apt-get source to get the source, so > it does have the same version with the object. I did: > tar zxvf otp_src_R13B03.tar.gz > cd otp_src_R13B03 > ./configure; make The binary files of interest are bin/x86_64-unknown-linux-gnu/beam and erts/emulator/obj/x86_64-unknown-linux-gnu/opt/plain/erl_goodfit_alloc.o. > > You can get a stack dump from the crash by attaching gdb to the > > soon-to-crash beam process. Now instead of being terminated gdb will > > get control of the process and you should be able to print a stack > > trace with bt or where. (This does require that there's a sufficient > > time window from the start of the application to the crash.) > > =20 > I've make a core dump 4 seconds before it crash, as mentioned above,=20 > because don't get the right symbols, it just with some quesion-marks: > > Core was generated by `/usr/lib/erlang/erts-5.7.2/bin/beam'. > #0 0x00007f0a28ecd5a9 in ?? () > (gdb) whe > #0 0x00007f0a28ecd5a9 in ?? () > #1 0x0000000000000000 in ?? () > (gdb) A core dump from a time point before the crash is useless. Either get a core dump from the crash itself (execute `ulimit -c unlimited' in bash before running the test), or attach gdb, continue the process, and wait for gdb to receive control when the crash occurs. > Look back the initial error: > > segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in > beam[400000+174000] > > Would you mind tell me what's the meaning of -ip-/-sp-, and what does > -error 4- means? IP is the Instruction Pointer aka Program Counter. SP is the stack pointer. The 'error 4' is a low-level error code which is irrelevant for us. From wde@REDACTED Wed May 26 20:07:08 2010 From: wde@REDACTED (wde) Date: Wed, 26 May 2010 20:07:08 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? Message-ID: <20100526180722.B4A7470000B2@msfrf2208.sfr.fr> Author: John Haugeland MIT LICENSE svn://crunchyd.com/scutil/erl/src/sc_random.erl shuffle(List) -> WeightedAndShuffled = lists:map( fun(Item) -> { random:uniform(), Item } end, List ), { _, SortedAndDeweighted } = lists:unzip(lists:keysort(1, WeightedAndShuffled)), SortedAndDeweighted. >I had a method in Javascript that was based on the Fisher-Yates shuffle >that I tried converting to Erlang. It essentially zips the given list >with a sequence 1-N, and uses a combination of lists:nth/2 and >proplists:delete/2 to create the shuffled list. I've not looked into the >entropy or bias as I've just thrown this together but the simple testing >seems to be positive. > >%% unsort >-module(unsort) > >-export([shuffle/1]). > >shuffle(L) -> > Len = length(L), > PropL = lists:zip(lists:seq(1, Len), L), > shuffle(PropL, Len, []). > >shuffle([], _, L) -> lists:reverse(L); >shuffle([{_,V}], _, L) -> lists:reverse([V | L]); >shuffle([{_,V} | T]=Ps, Len, L) -> > case swap_idx(Len) of > 1 -> > shuffle(T, Len-1, [V | L]); > SwapIdx -> > {K1, V1} = lists:nth(SwapIdx, Ps), > shuffle(proplists:delete(K1, Ps), Len-1, [V1 | L]) > end. > >swap_idx(Len) -> > trunc(random:uniform() * Len) + 1. > >%% N length list >%% Create X shuffled lists >test(N, X) -> > crypto:start(), > L = lists:seq(1,N), > Md5s = lists:map(fun(_) -> > >string:to_upper(lists:flatten([io_lib:format("~2.16.0b", [Digest]) || ><> <= crypto:md5(shuffle:shuffle(L))])) > end, lists:seq(1,X)), > gb_trees:to_list( > lists:foldl(fun(Hash, Tree) -> > case gb_trees:lookup(Hash, Tree) of > {value, Count} -> > gb_trees:update(Hash, Count+1, Tree); > _ -> gb_trees:insert(Hash, 1, Tree) > end > end, gb_trees:empty(), Md5s)). > >%% Sample output >1> unsort:test(2, 100000). >[{"050D144172D916D0846F839E0412E929",50325}, > {"0CB988D042A7F28DD5FE2B55B3F5AC7A",49675}] >2> unsort:test(3, 100000). >[{"3BAC2889507C76C73913B5CACEB75B6D",16668}, > {"5289DF737DF57326FCDD22597AFB1FAC",16674}, > {"5D582C1341F4783E2FDE0F07D0AF9A06",16581}, > {"9DE228839B2CCC8D08071BCADA88CA6D",16797}, > {"A867F6E8EBD4E3A90C6084C9738B506B",16656}, > {"AE0F84A71C987607257309BF36DFB41A",16624}] >3> unsort:test(4, 100000). >[{"08D6C05A21512A79A1DFEB9D2A8F262F",4217}, > {"0A27E477F545F21001C975A0DA5826CF",4244}, > {"0AA9093A8C91D9BAF3DDB40613BF6052",4092}, > {"0AFB83810CCDA4270CCBB36128D7258B",4152}, > {"0E4717467434E3A8AF70C101789434D4",4202}, > {"21C49141D9D5062E9C8A3FA8591507A7",4081}, > {"26E5799C9C28C4F750DE5289644CCBCB",4190}, > {"3044356A01283AC9BF0E83A69A76DAFA",4303}, > {"32E2F90B64F81FEDB13FD92F554194EF",4069}, > {"428C536DB5E72C45CF09CCD7A229D482",4185}, > {"46DEB0BCE99B407029591EC8858F8D76",4185}, > {"5DA92947151FE75785B296B4EE680265",4109}, > {"6F3FF712FAE51C9E6A34E5649B1DF983",4058}, > {"A9380E383BA79F5FCB90CF0F3A6F61C1",4164}, > {"C73CABEB6558ABA030BBA9CA49DCDD75",3996}, > {"C7E1C6D5AE3E1AC8B876F500F26BF37B",4132}, > {"CBEED65578A453CDA5B7F9899F33F63F",4124}, > {"DC7C46E4A1969FAF25ED7F53A0E7993B",4337}, > {"DE5FE0700986F246278524D88F24DB10",4200}, > {"DF2CEBAAE30811FFA20BA50D8E3EF365",4066}, > {"E182CD2D19E8C3555B7D63454FC2D1BB",4218}, > {"E64FEF4E93468D853B99662B25D37193",4240}, > {"E78F3DE3AAE140076643E5D6F8E4C4B9",4204}, > {"E94E7EE783C0AEE87D19D87F071FA2FD",4232}] >4> > > > >On 05/26/2010 12:09 PM, Bj?rn-Egil Dahlberg wrote: >> On 2010-05-26 17:14, Johan Montelius wrote: >>> On Wed, 26 May 2010 14:41:19 +0200, Bj?rn-Egil Dahlberg >>> wrote: >>> >>>> My suggestion is included in this mail, but my question is: does anyone >>>> have a faster, more generic and less memory consuming shuffle function? >>> >>> I haven't done any test but I would think the execution time will be >>> very much depending on the number of random:uniform/1 calls that you do. >>> As it is now you make a lot of calls but do not utilize the entropy. A >>> don't think a call to random:uniform(2) is much cheaper than >>> random:uniform(8098809898) nor do I know how well random:uniform is >>> implemented. >> >> You are of course completely right. There is a lot of unnecessary >> calls to random:uniform/1. One can always redefine the constraints for >> the fun. Instead of returning a number, return a binary. The fun is >> there to provide the user with a alternative random-function to the >> shuffle function, mainly for creating pseudo-random shuffles. >> shuffle/1 uses random:uniform/1 as default. >> >>> Here is an idea: >>> >>> generate a SHA-512 binary, use the binary bit by bit to divide the list >>> into two lists, if you run out of bits, generate a new pattern. Return >>> the remaining bit pattern and the two lists (this is similar to split in >>> qsort). Recursively apply to the first list and then the second list and >>> if we do it right we can avoid doing the last append (me think). >> >> +1, I like this idea. Easy to hook in crypto:rand_bytes/1 also. >> >> // Bj?rn-Egil >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > >-- >James Aimonetti >mobile: 314.809.6307 >work: 540.459.2220 >email: james.aimonetti@REDACTED >website: http://jamesaimonetti.com > > >________________________________________________________________ >erlang-questions (at) erlang.org mailing list. >See http://www.erlang.org/faq.html >To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rvirding@REDACTED Wed May 26 22:11:53 2010 From: rvirding@REDACTED (Robert Virding) Date: Wed, 26 May 2010 22:11:53 +0200 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: On 26 May 2010 18:42, Igor Ribeiro Sucupira wrote: > Since a dict consumes more space, the advantage of the proplist is the > size. Basically, the dict will give you a performance gain that's > irrelevant (unless you have many columns) and use more space. In that case you might as well use an orddict which is generally faster than an unsorted list, though not as fast as dict or gb_trees. You will find that the relative size difference between orddict/proplist and gb_trees decreases as the number of elements grows. > For example: > > 1> List = [{"firstname", "Igor"}, {"lastname", "Sucupira"}, {"gender", > "male"}, {"birthdate", "02/15/1981"}]. > [{"firstname","Igor"}, > ?{"lastname","Sucupira"}, > ?{"gender","male"}, > ?{"birthdate","02/15/1981"}] > 2> Dict= dict:from_list(List). > {dict,4,16,16,8,80,48, > ? ? ?{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > ? ? ?{{[["lastname",83,117,99,117,112,105,114,97], > ? ? ? ? ["gender",109,97,108,101]], > ? ? ? ?[], > ? ? ? ?[["birthdate",48,50,47,49,53,47,49,57,56,49]], > ? ? ? ?[],[],[],[],[],[],[],[],[], > ? ? ? ?[["firstname",73,103,111,114]], > ? ? ? ?[],[],[]}}} > 3> GB = gb_trees:from_orddict(orddict:from_list(List)). > {4, > ?{"gender","male", > ?{"firstname","Igor",{"birthdate","02/15/1981",nil,nil},nil}, > ?{"lastname","Sucupira",nil,nil}}} > > 4> erlang:byte_size(term_to_binary(List)). > 97 > 5> erlang:byte_size(term_to_binary(Dict)). > 195 > 6> erlang:byte_size(term_to_binary(GB)). > 125 > > 8> timer:tc(lists, keysearch, ["firstname", 1, List]). > {2,{value,{"firstname","Igor"}}} > 9> timer:tc(gb_trees, lookup, ["firstname", GB]). > {3,{value,"Igor"}} > 10> timer:tc(dict, find, ["firstname", Dict]). > {4,{ok,"Igor"}} > 11> timer:tc(lists, keysearch, ["firstname", 1, List]). > {3,{value,{"firstname","Igor"}}} > 12> timer:tc(gb_trees, lookup, ["firstname", GB]). > {3,{value,"Igor"}} > 13> timer:tc(dict, find, ["firstname", Dict]). > {4,{ok,"Igor"}} These times are really too small to hope to show anything when comparing them. You should run the tests 10k times for each one to get some relevant figures. How many columns do you think you will be using? Robert From johanmon@REDACTED Wed May 26 23:15:04 2010 From: johanmon@REDACTED (Johan Montelius) Date: Wed, 26 May 2010 23:15:04 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD47CE.1070601@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> <4BFD47CE.1070601@erix.ericsson.se> Message-ID: ok, kids are a sleep, here is my go. The random function can be replaced to just about anything. My only worries is that it is generating a lot of garbage. %% copy right, creative commons -module(unsort). -export([shuffle/1, shuffle/2]). shuffle(Is) when is_list(Is) -> shuffle(Is, generator(12345)). shuffle(Is, F) -> shuffle(Is, F, []). shuffle([], F, Sofar) -> {F, Sofar}; shuffle([I], F, Sofar) -> {F, [I|Sofar]}; shuffle(Is, F, Sofar) when is_list(Is), is_function(F) -> {F1, S1, S2} = split(Is, F, [], []), {F2, Q1} = shuffle(S1, F1, Sofar), shuffle(S2, F2, Q1). split([], F, L1, L2) -> {F, L1, L2}; split([H|T], F, L1, L2) -> case F() of {0, F1} -> split(T, F1, [H|L1], L2); {1, F1} -> split(T, F1, L1, [H|L2]) end. %% generator should return a function F, that when applied F() returns a tuple %% {R, F1} wher R is 0 or 1 and F1 a new function with the same property. generator(Seed) -> crypto:start(), fun() -> rbit([], <<>>, fun() -> rnd(<>) end) end. rbit([], <<>>, F) -> {Bin, F1} = F(), rbit([], Bin, F1); rbit([], Bin, F) -> <> = Bin, rbit([B1,B2,B3,B4,B5,B6,B7,B8], Rest, F); rbit([R|Rest], Bin, F) -> {R, fun() -> rbit(Rest, Bin, F) end}. rnd(Seed) -> Bin = crypto:md5(Seed), {Bin, fun() -> rnd(Bin) end}. -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From toby@REDACTED Thu May 27 02:47:03 2010 From: toby@REDACTED (Toby Thain) Date: Thu, 27 May 2010 10:47:03 +1000 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> On 27-May-10, at 2:58 AM, Jesper Louis Andersen wrote: > 2010/5/26 Bj?rn-Egil Dahlberg : >> I while back there were some talk about an unsort or shuffle >> operation for >> lists. I too was in need of this since it very convenient in >> testing, so I >> wrote one. > > Etorrent has one in its utils library as well: > > http://github.com/jlouis/etorrent/blob/master/src/etorrent_utils.erl#L101 > > It works by creating an analogy to shuffling a deck of cards and I am > somewhat sure it will uniformly shuffle It won't. Use Fisher-Yates or an equivalent method. http://en.wikipedia.org/wiki/Fisher?Yates_shuffle --Toby > (though I have not proved that > it is the case). The code is originally by Taylor Campbell if memory > serves. I also think this is an excellent candidate for a standard > library function. Getting it right is quite difficult. Getting it fast > and right is even more difficult. I also note that the > binary-generation trick ought to be usable on this one. > > In Etorrent it is rarely called and every place where it is used, the > client would work even if you omit the call. Hence, it has not seen > much testing. > > > -- > J. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From igorrs@REDACTED Thu May 27 04:25:52 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 26 May 2010 23:25:52 -0300 Subject: [erlang-questions] mnesia:transform_table In-Reply-To: References: Message-ID: On Wed, May 26, 2010 at 5:11 PM, Robert Virding wrote: > On 26 May 2010 18:42, Igor Ribeiro Sucupira wrote: >> Since a dict consumes more space, the advantage of the proplist is the >> size. Basically, the dict will give you a performance gain that's >> irrelevant (unless you have many columns) and use more space. > > In that case you might as well use an orddict which is generally Interesting... I hadn't noticed that an orddict is basically a proplist sorted by "keys". Since it's a list (and not an imperative array), I suppose you can't use binary search and should still need linear time (in the worst case) both to find and to insert/update an element, with constant time in the best case. And those are also the bounds for "normal" proplists. If that is the case, it seems to me that an orddict will only be more efficient (in practice) than a proplist for search misses, since on a proplist you always need to search the whole list to decide on a search miss. For most applications, I think search misses for fields on a database will not be very common, so maybe it doesn't make much difference in the end if you use an orddict or an unsorted proplist... > faster than an unsorted list, though not as fast as dict or gb_trees. > You will find that the relative size difference between > orddict/proplist and gb_trees decreases as the number of elements > grows. > >> For example: >> >> 1> List = [{"firstname", "Igor"}, {"lastname", "Sucupira"}, {"gender", >> "male"}, {"birthdate", "02/15/1981"}]. >> [{"firstname","Igor"}, >> ?{"lastname","Sucupira"}, >> ?{"gender","male"}, >> ?{"birthdate","02/15/1981"}] >> 2> Dict= dict:from_list(List). >> {dict,4,16,16,8,80,48, >> ? ? ?{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >> ? ? ?{{[["lastname",83,117,99,117,112,105,114,97], >> ? ? ? ? ["gender",109,97,108,101]], >> ? ? ? ?[], >> ? ? ? ?[["birthdate",48,50,47,49,53,47,49,57,56,49]], >> ? ? ? ?[],[],[],[],[],[],[],[],[], >> ? ? ? ?[["firstname",73,103,111,114]], >> ? ? ? ?[],[],[]}}} >> 3> GB = gb_trees:from_orddict(orddict:from_list(List)). >> {4, >> ?{"gender","male", >> ?{"firstname","Igor",{"birthdate","02/15/1981",nil,nil},nil}, >> ?{"lastname","Sucupira",nil,nil}}} >> >> 4> erlang:byte_size(term_to_binary(List)). >> 97 >> 5> erlang:byte_size(term_to_binary(Dict)). >> 195 >> 6> erlang:byte_size(term_to_binary(GB)). >> 125 >> >> 8> timer:tc(lists, keysearch, ["firstname", 1, List]). >> {2,{value,{"firstname","Igor"}}} >> 9> timer:tc(gb_trees, lookup, ["firstname", GB]). >> {3,{value,"Igor"}} >> 10> timer:tc(dict, find, ["firstname", Dict]). >> {4,{ok,"Igor"}} >> 11> timer:tc(lists, keysearch, ["firstname", 1, List]). >> {3,{value,{"firstname","Igor"}}} >> 12> timer:tc(gb_trees, lookup, ["firstname", GB]). >> {3,{value,"Igor"}} >> 13> timer:tc(dict, find, ["firstname", Dict]). >> {4,{ok,"Igor"}} > > These times are really too small to hope to show anything when Yeah... I know. I thought I had already made my point before (that any difference would be irrelevant if the number of fields is small, considering you already had to search and retrieve the list of fields from the database) and didn't make an effort to provide adequate evidence. ;-) It may also be a good idea to compile the tests you run, if they are more complex than those above, as Robert made me learn in the thread below. :-) http://forum.trapexit.org/viewtopic.php?p=49172&sid=5b490b5c8ebee8caadbfc0c5b808c07d Best regards. Igor. > comparing them. You should run the tests 10k times for each one to get > some relevant figures. > > How many columns do you think you will be using? > > Robert From ok@REDACTED Thu May 27 05:23:33 2010 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 27 May 2010 15:23:33 +1200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD16EF.6060206@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> The code that was posted is pleasantly fast but has a critical defect. Suppose the input is [A,B,C,D, E,F,G,H] then the output will be randperm([A,B,C,D]) ++ randperm([E,F,G,H]) or randperm([E,F,G,H]) ++ randperm([A,B,C,D]) It will never interleave the two blocks. The simplest method I can think of for Erlang is random_permutation(L) -> unwrap(lists:sort(wrap(L, [])), []). wrap([X|Xs], L) -> wrap(Xs, [{random:uniform(),X}|L]); wrap([], L) -> L. unwrap([{_,X}|L], Xs) -> unwrap(L, [X|Xs]); unwrap([], Xs) -> Xs. This is a bit slower, but it does have every possible permutation as a logically possible output, and under the usual naive assumptions about RNGs they are all equally likely. (Under somewhat less naive assumptions if you have an k-bit seed you can generate at most 2**k different random sequences, and if 2**k < n! not all permutations will really be possible. But that applies to anything that uses a limited number of physically random bits.) From luismarianoguerra@REDACTED Thu May 27 08:10:26 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Thu, 27 May 2010 03:10:26 -0300 Subject: error in documentation about port drivers Message-ID: I was following this documentation and I had problems making it work: http://www.erlang.org/doc/tutorial/c_portdriver.html the problem is: unix> gcc -o exampledrv -fpic -shared complex.c port_driver.c should be unix> gcc -o example_drv.so -fpic -shared complex.c port_driver.c that's all From luismarianoguerra@REDACTED Thu May 27 08:11:20 2010 From: luismarianoguerra@REDACTED (Mariano Guerra) Date: Thu, 27 May 2010 03:11:20 -0300 Subject: error in documentation about port drivers In-Reply-To: References: Message-ID: On Thu, May 27, 2010 at 3:10 AM, Mariano Guerra wrote: > I was following this documentation and I had problems making it work: > > http://www.erlang.org/doc/tutorial/c_portdriver.html > > the problem is: > > unix> gcc -o exampledrv -fpic -shared complex.c port_driver.c > > should be > > unix> gcc -o example_drv.so -fpic -shared complex.c port_driver.c > > that's all no windows to test but I think the line: windows> cl -LD -MD -Fe exampledrv.dll complex.c port_driver.c should be windows> cl -LD -MD -Fe example_drv.dll complex.c port_driver.c From johanmon@REDACTED Thu May 27 09:39:57 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 09:39:57 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, 27 May 2010 02:47:03 +0200, Toby Thain wrote: > It won't. Use Fisher-Yates or an equivalent method. > http://en.wikipedia.org/wiki/Fisher?Yates_shuffle Fisher-Yates (or the Knuth version) uses teh fact that you can access elements in an array and do the shuffling in O(n). If we have a list to shuffle the same algorithm will be O(n2) since we have to run down the list do find the element we want to shuffle. If we are working with lists I think any solutions is a version of merge-sort or quick-sort and have a O(n*lg(n)) complexity. -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From huss01@REDACTED Thu May 27 10:10:53 2010 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Thu, 27 May 2010 10:10:53 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, May 27, 2010 at 09:39, Johan Montelius wrote: > On Thu, 27 May 2010 02:47:03 +0200, Toby Thain > wrote: > >> It won't. Use Fisher-Yates or an equivalent method. >> http://en.wikipedia.org/wiki/Fisher?Yates_shuffle > > Fisher-Yates (or the Knuth version) uses teh fact that you can access > elements in an array and do the shuffling in O(n). If we have a list to > shuffle the same algorithm will be O(n2) since we have to run down the list > do find the element we want to shuffle. > > If we are working with lists I think any solutions is a version of > merge-sort or quick-sort and have a O(n*lg(n)) complexity. > Indeed. The random merge that has been shown here earlier can be corrected by replacing the coin flip with a choice that takes into account the length of the lists: -module(unsort). -export([unsort/1]). unsort(L) -> unsort(L, length(L)). unsort(L, Len) when Len < 2 -> L; unsort(L, Len) -> L1_len = Len div 2, L2_len = Len - L1_len, {L1, L2} = lists:split(L1_len, L), randmerge(unsort(L1, L1_len), L1_len, unsort(L2, L2_len), L2_len). randmerge([], 0, L2, _) -> L2; randmerge(L1, _, [], 0) -> L1; randmerge(L1, L1_len, L2, L2_len) -> case random:uniform(L1_len + L2_len) of N when N =< L1_len -> [hd(L1) | randmerge(tl(L1), L1_len - 1, L2, L2_len)]; _ -> [hd(L2) | randmerge(L1, L1_len, tl(L2), L2_len - 1)] end. /H?kan From johanmon@REDACTED Thu May 27 10:33:01 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 10:33:01 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, 27 May 2010 10:10:53 +0200, H?kan Huss wrote: > randmerge([], 0, L2, _) -> > L2; > randmerge(L1, _, [], 0) -> > L1; > randmerge(L1, L1_len, L2, L2_len) -> > case random:uniform(L1_len + L2_len) of > N when N =< L1_len -> > [hd(L1) | randmerge(tl(L1), L1_len - 1, L2, L2_len)]; > _ -> > [hd(L2) | randmerge(L1, L1_len, tl(L2), L2_len - 1)] > end. Why use the length of the list? What is the advantage of using a coin-flip? randmerge([], L2) -> L2; randmerge(L1, []) -> L1; randmerge(L1, L2) -> case random:uniform(2) of 1 -> [hd(L1) | randmerge(tl(L1), L2)]; 2 -> [hd(L2) | randmerge(L1, tl(L2))] end. -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From johanmon@REDACTED Thu May 27 11:52:52 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 11:52:52 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: > Contrast this with the version that takes into account the length of > both lists. > > Julian Very interesting, I didn't think about that. The same problem does not occur in my qsort version does it? Come to think of it I don't think my version is guaranteed to terminate. If we are extremely unlucky the split will generate one empty lists and one list with all elements in it :-0 well, for those of you who like living on the edge here is a rewrite that (if it terminates:-) is quite quick. shuffle(L) -> shuffle(L, [], <<>>, rbit(12345), []). shuffle([], R, B, S, Sofar) -> {R, B, S, Sofar}; shuffle([I], R, B, S, Sofar) -> {R, B, S, [I|Sofar]}; shuffle(L, Rnd, Bin, Seed, Sofar) -> {R1, B1, S1, L1, L2} = split(L, Rnd, Bin, Seed, [], []), {R2, B2, S2, Q1} = shuffle(L1, R1, B1, S1, Sofar), shuffle(L2, R2, B2, S2, Q1). split([], Rnd, Bin, Seed, L1, L2) -> {Rnd, Bin, Seed, L1, L2}; split([H|T], [R|Rnd], Bin, Seed, L1, L2) -> case R of 0 -> split(T, Rnd, Bin, Seed, [H|L1], L2); 1 -> split(T, Rnd, Bin, Seed, L1, [H|L2]) end; split(L, [], Bin, Seed, L1, L2) -> {Rnd, Rest, New} = rbit(Bin, Seed), split(L, Rnd, Rest, New, L1, L2). rbit(Seed) -> crypto:start(), <>. rbit(<<>>, Seed) -> Bin = crypto:md5(Seed), rbit(Bin, Bin); rbit(<>, Seed) -> {[B1,B2,B3,B4,B5,B6,B7,B8], Rest, Seed}. -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From serge@REDACTED Thu May 27 12:18:56 2010 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 27 May 2010 06:18:56 -0400 Subject: ERL_FLOAT_EXT Message-ID: <4BFE4710.70107@aleynikov.org> A while ago I submitted a patch for ei to support IEEE 754 double encoding/decoding compliant with ERL_FLOAT_EXT format. http://www.erlang.org/cgi-bin/ezmlm-cgi/3/256 Is it possible for it to get included in the distribution? The rationale is that the emulator supports compact encoding of doubles in external binary format, but ei doesn't. Serge From julian.tibble@REDACTED Thu May 27 12:47:06 2010 From: julian.tibble@REDACTED (Julian Tibble) Date: Thu, 27 May 2010 11:47:06 +0100 Subject: Fwd: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: Apologies, the e-mail below did not get through to the list the first time I sent it. ------------------ > Why use the length of the list? What is the advantage of using a coin-flip? > > randmerge([], L2) -> > ? L2; > randmerge(L1, []) -> > ? L1; > randmerge(L1, L2) -> > ? case random:uniform(2) of > ? ? ? 1 -> > ? ? ? ? ? [hd(L1) | randmerge(tl(L1), L2)]; > ? ? ? 2 -> > ? ? ? ? ? [hd(L2) | randmerge(L1, tl(L2))] > ? end. The function "randmerge(L,R)" should be fair - that is, it should produce all interleavings of L and R with equal likelihood, but this is not true with the coin-flip version. To see why, consider computing randmerge([1,3], [2,4]) At each step, randmerge can choose the left list or the right list. Therefore the choices are: L L - 1,3,2,4 L R L - 1,2,3,4 L R R - 1,2,4,3 R L L - 2,1,3,4 R L R - 2,1,4,3 R R - 2,4,1,3 (note, after "L L" the left list is empty so there is no further random choice. ?similarly for "R R") When using a coin-flip, the top and bottom cases both have probability 1/4. All the other cases have probability 1/8. Therefore randmerge is not fair. Contrast this with the version that takes into account the length of both lists. Julian From ivan@REDACTED Thu May 27 11:31:21 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 27 May 2010 10:31:21 +0100 Subject: How to convert a binary to a list of 2 byte ints Message-ID: <4BFE3BE9.5080800@llaisdy.com> Dear All binary_to_list/1 converts a binary to a list of integers, mapping each byte onto an integer. The data I have, I need to map each pair of bytes onto an integer (signed, little-endian). e.g.: 1> b2i(<<18,255,39,255,73,255,90,255,80,255>>). [-238, -217, -183, -166, -176] I am learning erlang. This code returns a list of binaries (each of length 2 bytes), but I haven't been able to make the last step of converting to ints. b2i(Data) -> X = b2iacc(Data, []). io:format("d2c: ~p~n", [X]), X. b2iacc(<>, Out) -> X = <>, b2iacc(R, [X | Out]); b2iacc(<<>>, Out) -> lists:reverse(Out). Am I nearly there, or am I barking up the wrong tree? With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From gleber.p@REDACTED Thu May 27 12:58:02 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 27 May 2010 12:58:02 +0200 Subject: [erlang-questions] How to convert a binary to a list of 2 byte ints In-Reply-To: <4BFE3BE9.5080800@llaisdy.com> References: <4BFE3BE9.5080800@llaisdy.com> Message-ID: You can use binary comprehension: 4> X = <<18,255,39,255,73,255,90,255,80,255>>. <<18,255,39,255,73,255,90,255,80,255>> 5> [ H || <> <= X ]. [-238,-217,-183,-166,-176] On Thu, May 27, 2010 at 11:31, Ivan Uemlianin wrote: > Dear All > > binary_to_list/1 converts a binary to a list of integers, mapping each byte > onto an integer. ?The data I have, I need to map each pair of bytes onto an > integer (signed, little-endian). > > e.g.: > > ? ?1> b2i(<<18,255,39,255,73,255,90,255,80,255>>). > > ? ?[-238, -217, -183, -166, -176] > > I am learning erlang. > > This code returns a list of binaries (each of length 2 bytes), but I haven't > been able to make the last step of converting to ints. > > ? ?b2i(Data) -> > ? ? ? ?X = b2iacc(Data, []). > ? ? ? ?io:format("d2c: ~p~n", [X]), > ? ? ? ?X. > > ? ?b2iacc(<>, Out) -> > ? ? ? ?X = <>, > ? ? ? ?b2iacc(R, [X | Out]); > ? ?b2iacc(<<>>, Out) -> > ? ? ? ?lists:reverse(Out). > > Am I nearly there, or am I barking up the wrong tree? > > With thanks and best wishes > > Ivan > > -- > ============================================================ > Ivan A. Uemlianin > Speech Technology Research and Development > > ? ? ? ? ? ? ? ? ? ?ivan@REDACTED > ? ? ? ? ? ? ? ? ? ? www.llaisdy.com > ? ? ? ? ? ? ? ? ? ? ? ? llaisdy.wordpress.com > ? ? ? ? ? ? ? ? ? ? www.linkedin.com/in/ivanuemlianin > > ? ?"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > ? ? ? ? ? ? ? ? ? ? (Schiller, Beethoven) > ============================================================ > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From huss01@REDACTED Thu May 27 13:12:52 2010 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Thu, 27 May 2010 13:12:52 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, May 27, 2010 at 11:52, Johan Montelius wrote: > >> Contrast this with the version that takes into account the length of both >> lists. >> >> Julian > > Very interesting, I didn't think about that. The same problem does not occur > in my qsort version does it? > Actually, the same problem will occur for any version that uses only unbiased coin-flips. To see why, construct a decision tree where every internal node splits according to the outcome of a coin flip. The leaves of this tree will then be the permutation generated by that sequence of coin flips. Now, the probability of getting each permutation is 1/2 to the m:th power, where m is the length of the path to the permutation from the root. In order for the shuffle to be fair, these must be equal, which means that the tree must be full and complete. However, since there are n! possible permutations, this can only happen if n! = 2^m, which is not the case (unless n<=2). Regards, /H?kan From ivan@REDACTED Thu May 27 13:15:43 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 27 May 2010 12:15:43 +0100 Subject: [erlang-questions] How to convert a binary to a list of 2 byte ints In-Reply-To: References: <4BFE3BE9.5080800@llaisdy.com> Message-ID: <4BFE545F.6040304@llaisdy.com> Dear Gleb That is fantastic, thanks! I had tried it with bitstring comprehensions but to no avail. Here you are using the bitstring generator <= in a list comprehension. Cool! Best wishes Ivan On 27/05/2010 11:58, Gleb Peregud wrote: > You can use binary comprehension: > > 4> X =<<18,255,39,255,73,255,90,255,80,255>>. > <<18,255,39,255,73,255,90,255,80,255>> > 5> [ H ||<> <= X ]. > [-238,-217,-183,-166,-176] > -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From hans.bolinder@REDACTED Thu May 27 13:22:20 2010 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Thu, 27 May 2010 13:22:20 +0200 Subject: [erlang-questions] pg2 still busted in R13B04 In-Reply-To: References: Message-ID: <19454.21996.250478.654248@ornendil.du.uab.ericsson.se> [Rick Pettit:] > A while ago someone had posted that pg2 appeared to be broken as of R13B03. > > I am doing some testing with R13B04, and am finding the same to be true. > > Namely, with R13B04, it is possible for the same process to appear in a > group many times (even if that process only added itself once). This leads > to all sorts of problems, and makes it impossible to use pg2 with this > release. Could you please try the following patch? The patch should correct one bug introduced in R13B03. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson --- /usr/local/otp/releases/otp_beam_linux_sles10_x64_r13b_latest/lib/kernel-2.13.5.1/src/pg2.erl 2010-03-09 12:45:05.000000000 +0100 +++ /clearcase/otp/erts/lib/kernel/src/pg2.erl 2010-05-27 12:53:02.000000000 +0200 @@ -251,7 +251,9 @@ %%% Pid is a member of group Name. store(List) -> - _ = [assure_group(Name) andalso [join_group(Name, P) || P <- Members] || + _ = [(assure_group(Name) + andalso + [join_group(Name, P) || P <- Members -- group_members(Name)]) || [Name, Members] <- List], ok. From johanmon@REDACTED Thu May 27 15:07:45 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 15:07:45 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, 27 May 2010 13:12:52 +0200, H?kan Huss wrote: > However, since there are n! possible permutations, this can > only happen if n! = 2^m, which is not the case (unless n<=2). Hmm, but m in the split case is n*lg(n) (assuming the splits are well balanced) and 2^(n*lg(n)) is a quite big number. 2^(n*lg(n)) = (2^lg(n))^n = n^n > n! Or am I totally lost (known to have been) Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From egil@REDACTED Thu May 27 15:12:24 2010 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 27 May 2010 15:12:24 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> References: <4BFD16EF.6060206@erix.ericsson.se> <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> Message-ID: <4BFE6FB8.2040607@erix.ericsson.se> On 2010-05-27 05:23, Richard O'Keefe wrote: > The code that was posted is pleasantly fast but has a > critical defect. Suppose the input is > [A,B,C,D, E,F,G,H] > then the output will be > randperm([A,B,C,D]) ++ randperm([E,F,G,H]) > or > randperm([E,F,G,H]) ++ randperm([A,B,C,D]) > It will never interleave the two blocks. I had a hunch and why I voiced my second concern. One can see this when inspecting a shuffled list with a sequence of number. Groupings will appear in the list and that triggered my single neuron dedicated for detecting order at least. > > The simplest method I can think of for Erlang is > > random_permutation(L) -> > unwrap(lists:sort(wrap(L, [])), []). > > wrap([X|Xs], L) -> wrap(Xs, [{random:uniform(),X}|L]); > wrap([], L) -> L. > > unwrap([{_,X}|L], Xs) -> unwrap(L, [X|Xs]); > unwrap([], Xs) -> Xs. > > This is a bit slower, but it does have every possible > permutation as a logically possible output, and under > the usual naive assumptions about RNGs they are all > equally likely. (Under somewhat less naive assumptions > if you have an k-bit seed you can generate at most 2**k > different random sequences, and if 2**k < n! not all > permutations will really be possible. But that applies > to anything that uses a limited number of physically > random bits.) Is there no need for assigning unique numbers to the list when sorting it? I guess it boils down to how fair we want the list to be unsorted (or how one wants to word it). // Bj?rn-Egil From rpettit@REDACTED Thu May 27 15:36:15 2010 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 27 May 2010 08:36:15 -0500 Subject: How erlang processes get mapped to OS threads with SMP erlang Message-ID: <58aef0aeb6de5bf1723bb590f11a4a31.squirrel@squirrelmail.vail> I've seen R13B-03 SMP erlang under load on a solaris system with multiple cores hit one core particularly hard for just a few seconds before load gets evenly distributed across all available cores as I would expect. By hard, I mean one CPU drops to 0% idle for a few seconds before load becomes very evenly spread across all available cores. Are there documents available which describe how erlang processes get mapped to OS threads with SMP erlang? Does this sound like something I might want to tune at the OS level, or on the erlang side? (I've already started work on throttling the work done by SMP erlang so that the distribution of load across cores can happen more smoothly, but am looking for any other pointers on how to address this problem). On solaris I've managed to isolate this SMP erlang node by placing it in its own processor set, but I would rather not have to restrict it to a subset of available cores (which are then unavailable to other processes running on the server). I'm happy to RTFM if someone could point me in the right direction for this particular problem. Thanks, -Rick From mevans@REDACTED Thu May 27 15:48:45 2010 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 27 May 2010 09:48:45 -0400 Subject: Process Monitoring Message-ID: Hi I asked this question a while ago, but no takers. I wish to monitor multiple processes on a remote card. Would the best approach be to do erlang:monitor/2, or do erlang:monitor_node/2 and write a simple (local) mechanism to alert interested parties locally? I guess the bigger question is: Is there any overhead of doing remote erlang:monitor/2 calls? Since gen_server uses it my guess is no. Thanks Matt From huss01@REDACTED Thu May 27 16:23:03 2010 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Thu, 27 May 2010 16:23:03 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, May 27, 2010 at 15:07, Johan Montelius wrote: > On Thu, 27 May 2010 13:12:52 +0200, H?kan Huss wrote: > >> However, since there are n! possible permutations, this can >> only happen if n! = 2^m, which is not the case (unless n<=2). > > Hmm, but m in the split case is n*lg(n) (assuming the splits are well > balanced) and 2^(n*lg(n)) is a quite big number. 2^(n*lg(n)) = (2^lg(n))^n = > n^n ?> n! ?Or am I totally lost (known to have been) > The reasoning doesn't in any way relate to the specifics of the algorithm, eg., whether splits are done or not. In general, what we are doing is mapping a sequence of random events to a specific permutation of the original list. Using coin flips, the random sequence could be heads, tails, tails, heads, tails,... Each such sequence will yield a specific permutation of the original list. For instance, for a three-element list [a,b,c] we could provide the following mapping (H: heads, T: tails): HH -> [a,b,c] HTH -> [a,c,b] HTT -> [b,a,c] THH -> [b,c,a] THT -> [c,a,b] TT -> [c,b,a] This can be displayed as a tree with edges that each correspond to an H or a T, and the leaves are the resulting permutations. Regardless of the details of the algiorithm, if it uses coin flips it can be depicted using such a tree. Since the probability of choosing a specific branch in this tree is 1/2, the probability of obtaining any specific permutation is (1/2)^m where m is the path length from the root of the tree to the permutation in question. In the above example, m is 2 for the first and the last permutation and 3 for the others. Thus, it is not a fair shuffle. In order for it to be fair, all path lengths need to be equal, but for this to be possible the number of leaves must be an exact power of 2. But since the leaves are the permutations, there are n! of them. This is never a power of two unless n <= 2. This means that there will be permutations with different path lengths, and thus there are permutations which are more likely than others. The only way to fix this is to make sure that the edges do not have the same probability of being taken. This is what the merge version I provided earlier did. But then you get an algorithm which is not based on coin flipping. (There is another possibility, of course, and that is to add dummy leaves to fill out the tree. In the above example, you can replace HH -> [a,b,c] with HHH -> [a,b,c] HHT -> invalid and TT -> [c,b,a] with TTH -> invalid, TTT -> [c,b,a]. If you also add the rule that hitting an invalid combination causes the algorithm to restart you will get a fair shuffle. However, applying this to an actual algorithm isn't necessarily easy. You also lose any deterministic time bounds since the algorithm may restart indefinitely). Regards, /H?kan From hd2010@REDACTED Thu May 27 16:24:13 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 16:24:13 +0200 Subject: Erlvolt Erlang VoltDB API Message-ID: <4BFE808D.50902@eonblast.com> Hi list, I am writing an API for VoltDB and critique would be welcome. - browse source via http://source.voltdb.com/browse/ClientAPI - svn download using svn checkout http://svnmirror.voltdb.com/ClientAPI/erlang/trunk VoltDB is the newest creation of Mike Stonebraker, a highly specialized OLTP database, optimized for high throughput. http://voltdb.com/product -- *Henning Diedrich* CEO Eonblast Corporation hdiedrich@REDACTED +1.404.418.5002 w www.eonblast.com This email contains confidential and/or privileged information. If you are not the intended recipient (or have received this email in error) please notify the sender immediately and destroy this email. Any unauthorized copying, disclosure or distribution of the material in this email is prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 4856 bytes Desc: not available URL: From johanmon@REDACTED Thu May 27 16:30:12 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 16:30:12 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: On Thu, 27 May 2010 16:23:03 +0200, H?kan Huss wrote: > If you also add the rule that hitting > an invalid combination causes the algorithm to restart you will get a > fair shuffle. However, applying this to an actual algorithm isn't > necessarily easy. You also lose any deterministic time bounds since > the algorithm may restart indefinitely). > Regards, > /H?kan Thanks for the explanation. When I first read your comment I thought that you meant that we did not have enough coin flips but the problem is of course that we use use too many. I realized this when I added a special case for shuffling three elements and as you point out I had some cases where I had to throw flips away. Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From hd2010@REDACTED Thu May 27 16:31:00 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 16:31:00 +0200 Subject: list_element() Message-ID: <4BFE8224.3020407@eonblast.com> Hi list, I need to access a list element by index. I wrote a function but it looks most clumsy. I am looking for maximal performance. Should I instead convert the list to a tuple and use element()? The lists are ~ 10 to 30 elements in length. (<-- not an attempt to define a frame). For curiosity, does the below tail-recurse correctly? listOrd(_, []) -> nil; listOrd(Searched, List) when is_list(List) -> listOrd(Searched, List, 1). listOrd(_, [], _) -> nil; listOrd(Searched, [ Element | Tail ], Count) -> case Searched == Element of true -> Count; _ -> listOrd(Searched, Tail, Count + 1) end. Thanks for your time, Henning From max.lapshin@REDACTED Thu May 27 16:31:19 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 27 May 2010 18:31:19 +0400 Subject: [erlang-questions] Erlvolt Erlang VoltDB API In-Reply-To: <4BFE808D.50902@eonblast.com> References: <4BFE808D.50902@eonblast.com> Message-ID: On Thu, May 27, 2010 at 6:24 PM, Henning Diedrich wrote: > Hi list, > > I am writing an API for VoltDB and critique would be welcome. > > - browse source via http://source.voltdb.com/browse/ClientAPI > > - svn download using svn checkout http://svnmirror.voltdb.com/ClientAPI/erlang/trunk > > http://svnmirror.voltdb.com/ClientAPI/erlang/trunk practically it is much more convenient to keep opensource on github.combecause of its "fork" button. From karol.skocik@REDACTED Thu May 27 16:38:51 2010 From: karol.skocik@REDACTED (karol skocik) Date: Thu, 27 May 2010 16:38:51 +0200 Subject: [erlang-questions] list_element() In-Reply-To: <4BFE8224.3020407@eonblast.com> References: <4BFE8224.3020407@eonblast.com> Message-ID: lists:nth(1, [1,2,3]). On Thu, May 27, 2010 at 4:31 PM, Henning Diedrich wrote: > Hi list, > > I need to access a list element by index. > > I wrote a function but it looks most clumsy. > > I am looking for maximal performance. Should I instead convert the list to a > tuple and use element()? > > The lists are ~ 10 to 30 elements in length. (<-- not an attempt to define a > frame). > > For curiosity, does the below tail-recurse correctly? > > ? ? ? listOrd(_, []) -> nil; > > ? ? ? listOrd(Searched, List) when is_list(List) -> > > ? ? ? ? ? listOrd(Searched, List, 1). > > ? ? ? listOrd(_, [], _) -> nil; > > ? ? ? listOrd(Searched, [ Element | Tail ], Count) -> > > ? ? ? ? ? case Searched == Element of > ? ? ? ? ? ? ? true -> Count; > ? ? ? ? ? ? ? _ -> listOrd(Searched, Tail, Count + 1) > ? ? ? ? ? end. > > > > Thanks for your time, > Henning > From ivan@REDACTED Thu May 27 16:40:57 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 27 May 2010 15:40:57 +0100 Subject: [erlang-questions] list_element() In-Reply-To: <4BFE8224.3020407@eonblast.com> References: <4BFE8224.3020407@eonblast.com> Message-ID: <4BFE8479.4010402@llaisdy.com> Dear Henning Do you mean you need to access the nth element of a list? Is lists:nth/2 no good? http://www.erlang.org/doc/man/lists.html#nth-2 Best wishes Ivan On 27/05/2010 15:31, Henning Diedrich wrote: > Hi list, > > I need to access a list element by index. > > I wrote a function but it looks most clumsy. > > I am looking for maximal performance. Should I instead convert the > list to a tuple and use element()? > > The lists are ~ 10 to 30 elements in length. (<-- not an attempt to > define a frame). > > For curiosity, does the below tail-recurse correctly? > > listOrd(_, []) -> nil; > > listOrd(Searched, List) when is_list(List) -> > > listOrd(Searched, List, 1). > > listOrd(_, [], _) -> nil; > > listOrd(Searched, [ Element | Tail ], Count) -> > > case Searched == Element of > true -> Count; > _ -> listOrd(Searched, Tail, Count + 1) > end. > > > > Thanks for your time, > Henning > -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From hd2010@REDACTED Thu May 27 16:47:21 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 16:47:21 +0200 Subject: Cost of Copy Message-ID: <4BFE85F9.3010705@eonblast.com> I am still honestly marveling about this. In case anyone finds time to answer. Especially on the theory that more deeply nested structures will be faster to mutate. As fewer internal 'sibling'-pointers will have to be copied. http://www.trapexit.org/forum/viewtopic.php?p=55001 Thanks, Henning From hd2010@REDACTED Thu May 27 16:50:03 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 16:50:03 +0200 Subject: [erlang-questions] list_element() In-Reply-To: <4BFE8479.4010402@llaisdy.com> References: <4BFE8224.3020407@eonblast.com> <4BFE8479.4010402@llaisdy.com> Message-ID: <4BFE869B.2060407@eonblast.com> No, look the index up by the content. That's a misleading title. Ivan Uemlianin wrote: > Dear Henning > > Do you mean you need to access the nth element of a list? Is > lists:nth/2 no good? > > http://www.erlang.org/doc/man/lists.html#nth-2 > > Best wishes > > Ivan > > On 27/05/2010 15:31, Henning Diedrich wrote: >> Hi list, >> >> I need to access a list element by index. >> >> I wrote a function but it looks most clumsy. >> >> I am looking for maximal performance. Should I instead convert the >> list to a tuple and use element()? >> >> The lists are ~ 10 to 30 elements in length. (<-- not an attempt to >> define a frame). >> >> For curiosity, does the below tail-recurse correctly? >> >> listOrd(_, []) -> nil; >> >> listOrd(Searched, List) when is_list(List) -> >> >> listOrd(Searched, List, 1). >> >> listOrd(_, [], _) -> nil; >> >> listOrd(Searched, [ Element | Tail ], Count) -> >> >> case Searched == Element of >> true -> Count; >> _ -> listOrd(Searched, Tail, Count + 1) >> end. >> >> >> >> Thanks for your time, >> Henning >> > > From chandrashekhar.mullaparthi@REDACTED Thu May 27 16:58:44 2010 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 27 May 2010 15:58:44 +0100 Subject: [erlang-questions] Process Monitoring In-Reply-To: References: Message-ID: On 27 May 2010 14:48, Evans, Matthew wrote: > Hi > > I asked this question a while ago, but no takers. > > I wish to monitor multiple processes on a remote card. Would the best > approach be to do erlang:monitor/2, or do erlang:monitor_node/2 and write a > simple (local) mechanism to alert interested parties locally? > > I guess the bigger question is: Is there any overhead of doing remote > erlang:monitor/2 calls? Since gen_server uses it my guess is no. > > I would use erlang:monitor/2 if only to save myself the effort of then implementing a local mechanism, however simple. cheers Chandru From rvirding@REDACTED Thu May 27 18:30:27 2010 From: rvirding@REDACTED (Robert Virding) Date: Thu, 27 May 2010 18:30:27 +0200 Subject: [erlang-questions] list_element() In-Reply-To: <4BFE8224.3020407@eonblast.com> References: <4BFE8224.3020407@eonblast.com> Message-ID: Your code is basically correct. Some points: - There is no real benefit in converting to a tuple first as you have to step over the elements anyway. - You should seriously consider how/what to return when you don't find the element. You return nil. This means that any caller will have to check return value to see if it was there, which means you may have to propagate error checking/handling. If it *should* there then maybe it is best just to signal an error if it not, by just skipping that clause. If you were putting them in a library then you could have two functions, one which returns {yes,Index} | no (say) and the other which just returns index and fails if not found. - The atom nil has no special significance in Erlang and is seldom used. Robert On 27 May 2010 16:31, Henning Diedrich wrote: > Hi list, > > I need to access a list element by index. > > I wrote a function but it looks most clumsy. > > I am looking for maximal performance. Should I instead convert the list to a > tuple and use element()? > > The lists are ~ 10 to 30 elements in length. (<-- not an attempt to define a > frame). > > For curiosity, does the below tail-recurse correctly? > > ? ? ? listOrd(_, []) -> nil; > > ? ? ? listOrd(Searched, List) when is_list(List) -> > > ? ? ? ? ? listOrd(Searched, List, 1). > > ? ? ? listOrd(_, [], _) -> nil; > > ? ? ? listOrd(Searched, [ Element | Tail ], Count) -> > > ? ? ? ? ? case Searched == Element of > ? ? ? ? ? ? ? true -> Count; > ? ? ? ? ? ? ? _ -> listOrd(Searched, Tail, Count + 1) > ? ? ? ? ? end. > > > > Thanks for your time, > Henning > From sgardell@REDACTED Thu May 27 19:01:02 2010 From: sgardell@REDACTED (Gardell, Steven) Date: Thu, 27 May 2010 13:01:02 -0400 Subject: Problem with io:fwrite/2 ? Message-ID: <6B3E29721F78364AA2C43697AFB15D9102D94F7C@sonusmail07.sonusnet.com> Running R13B04 under Linux RES 4. It seems that io:fwrite/2 is hanging "most of the time." With the following code segment: io:format("BEFORE FULL WRITE~n"), io:fwrite(CL#'CallLog'.file, "Calls: ~w Rate: ~w Trans: ~w %SlowAdd: ~.1f %SlowMod: ~.1f %SlowOther ~.1f %SlowTotal ~.1f %Slow2: ~.1f %Slow3: ~.1f Failed ~w Unexpected: ~w~n", [CL#'CallLog'.callCount, Rate, CL#'CallLog'.transCount, SlowAdd1, SlowMod1, SlowOther1, SlowTotal1, SlowTotal2, SlowTotal3, CL#'CallLog'.failedCount, CL#'CallLog'.unexpectedMsg]), io:format("BEFORE EMPTY LINE~n"), io:fwrite(CL#'CallLog'.file,"~n"), io:format("AFTER EMPTY LINE~n"). I see: 1> LOADTEST: Ready loadtest:onecall(twoterm4trans). Test: 5/27/2010 12:47:36 Rate: 1 Hold: 1 Calls:1 {go,1,1,0,1,1,twoterm4trans} BEFORE FULL WRITE BEFORE EMPTY LINE 2> The subsequent behavior of the program leads me to believe we never returned from fwrite(). From james.hague@REDACTED Thu May 27 20:09:55 2010 From: james.hague@REDACTED (James Hague) Date: Thu, 27 May 2010 13:09:55 -0500 Subject: [erlang-questions] list_element() In-Reply-To: References: <4BFE8224.3020407@eonblast.com> Message-ID: >The atom nil has no special significance in Erlang and is seldom used. One of those places is in the data structures gb_trees and gb_sets. I was surprised by that, as I can't recall seeing it elsewhere in the standard libraries. From james.hague@REDACTED Thu May 27 20:26:14 2010 From: james.hague@REDACTED (James Hague) Date: Thu, 27 May 2010 13:26:14 -0500 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4BFE85F9.3010705@eonblast.com> References: <4BFE85F9.3010705@eonblast.com> Message-ID: If you ignore message passing, then the copying/sharing issues in Erlang are just about the same as in Lisp and Scheme, so you may find some information that way. The short version is that some values are represented directly in a memory "cell," like atoms and (most) integers. Others are pointers to data stored elsewhere (list elements, or conses, which are two consecutive cells in memory) or tuples (N consecutive cells plus one for the size). Just by looking at code you can see where conses and tuples are created. If you use an existing "pointer" to data, then that data is not copied. Examples: B = {A, A, A} This creates a three element tuple (four memory cells). It doesn't matter if A is a list of a million elements or a tuple or the number 2. B just contains three cell-sized values (plus the size of the tuple). X = [57|List] This creates a single cons (2 cells). The first element is 57. The second is a pointer to List. So even though you've got two lists (X and List), all of the data is shared except for the first cons of X. From rpettit@REDACTED Thu May 27 21:11:06 2010 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 27 May 2010 14:11:06 -0500 Subject: [erlang-questions] How erlang processes get mapped to OS threads with SMP erlang In-Reply-To: <58aef0aeb6de5bf1723bb590f11a4a31.squirrel@squirrelmail.vail> References: <58aef0aeb6de5bf1723bb590f11a4a31.squirrel@squirrelmail.vail> Message-ID: On Thu, May 27, 2010 8:36 am, Rick Pettit wrote: > I've seen R13B-03 SMP erlang under load on a solaris system with multiple > cores hit one core particularly hard for just a few seconds before load > gets evenly distributed across all available cores as I would expect. By > hard, I mean one CPU drops to 0% idle for a few seconds before load > becomes very evenly spread across all available cores. > > Are there documents available which describe how erlang processes get > mapped to OS threads with SMP erlang? Does this sound like something I > might want to tune at the OS level, or on the erlang side? (I've already > started work on throttling the work done by SMP erlang so that the > distribution of load across cores can happen more smoothly, but am looking > for any other pointers on how to address this problem). > > On solaris I've managed to isolate this SMP erlang node by placing it in > its own processor set, but I would rather not have to restrict it to a > subset of available cores (which are then unavailable to other processes > running on the server). > > I'm happy to RTFM if someone could point me in the right direction for > this particular problem. Sorry for answering my own question, but the following seems to answer my question: http://erlang.2086793.n4.nabble.com/Some-facts-about-Erlang-and-SMP-td2108770.html In particular: "The schedulers in the Erlang VM are run on one OS-thread each and it is the OS that decides if the threads are executed on different Cores. Normally the OS will do this just fine and will also keep the thread on the same Core throughout the execution. " "The Erlang processes will be run by different schedulers because they are picked from a common run-queue by the first scheduler that becomes available. " -Rick From johanmon@REDACTED Thu May 27 21:58:12 2010 From: johanmon@REDACTED (Johan Montelius) Date: Thu, 27 May 2010 21:58:12 +0200 Subject: SV: [erlang-questions] list_element() In-Reply-To: <4BFE8224.3020407@eonblast.com> References: <4BFE8224.3020407@eonblast.com> Message-ID: <01155FE84766EC4D95AEA8E070E5EC81173CBCA566@MAIL01.ug.kth.se> If you do this over and over you might benefit from having the set sorted in a tree. That woudl give you lg(n) lookup but you would have to do a bit more work when you add new elements. Johan From hd2010@REDACTED Thu May 27 22:09:21 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 22:09:21 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> Message-ID: <4BFED171.1020100@eonblast.com> Thanks a lot James! am I right then to conclude that mutating something in a list will be the faster the more the data is spread out deeper over several levels? A = { B, C, D, E, F, G, H } will be slower to change than, say A = { { B, C }, { D, E }, {F, G, H } } because if I change e.g. B, only { B', C } will be created as new internal list of pointers, as opposed to the whole of { B', C, D, E, F, G, H }. Which should for larger lists and tuples, and frequent changes, start to make a difference? So this would be the fastes option? A = { { B }, { C }, { D }, { E }, { F }, { G }, { H } } And if the tuple may be expected to change in size (elements not only changeing but being added/deleted), this: A = { { { { B }, { C } }, { { D }, { E } } }, { { { F }, { G } }, { { H } } } } Maybe I am wrong that there may be a difference between tuples and lists, but is it the same for A = [ B, C, D, E, F, G, H ] slower to change than A = [ { B, C }, { D, E }, { F, G, H } ] ? Or are list-elements changed faster, in general, virtually 'in-place' as only one pointer is replaced? Hardly I guess, because the list itself must not be changed? So the whole new list must be copied anyway, just like a tuple. Thanks for your take, Henning James Hague wrote: > If you ignore message passing, then the copying/sharing issues in Erlang are > just about the same as in Lisp and Scheme, so you may find some information > that way. > > The short version is that some values are represented directly in a memory > "cell," like atoms and (most) integers. Others are pointers to data stored > elsewhere (list elements, or conses, which are two consecutive cells in > memory) or tuples (N consecutive cells plus one for the size). Just by > looking at code you can see where conses and tuples are created. If you use > an existing "pointer" to data, then that data is not copied. > > Examples: > > B = {A, A, A} > This creates a three element tuple (four memory cells). It doesn't matter if > A is a list of a million elements or a tuple or the number 2. B just > contains three cell-sized values (plus the size of the tuple). > > X = [57|List] > This creates a single cons (2 cells). The first element is 57. The second is > a pointer to List. So even though you've got two lists (X and List), all of > the data is shared except for the first cons of X. > > From hd2010@REDACTED Thu May 27 22:13:36 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 22:13:36 +0200 Subject: [erlang-questions] list_element() In-Reply-To: References: <4BFE8224.3020407@eonblast.com> Message-ID: <4BFED270.7040805@eonblast.com> Thank you Robert, I'll heed your two-versions advice and provide two functions there. Are null or undefined better than nil then, or is the desire to use such a non-value always a sign that an exception or error is in order? Thanks again, Henning Robert Virding wrote: > Your code is basically correct. Some points: > > - There is no real benefit in converting to a tuple first as you have > to step over the elements anyway. > > - You should seriously consider how/what to return when you don't > find the element. You return nil. This means that any caller will have > to check return value to see if it was there, which means you may have > to propagate error checking/handling. If it *should* there then maybe > it is best just to signal an error if it not, by just skipping that > clause. If you were putting them in a library then you could have two > functions, one which returns {yes,Index} | no (say) and the other > which just returns index and fails if not found. > > - The atom nil has no special significance in Erlang and is seldom used. > > Robert > > > On 27 May 2010 16:31, Henning Diedrich wrote: > >> Hi list, >> >> I need to access a list element by index. >> >> I wrote a function but it looks most clumsy. >> >> I am looking for maximal performance. Should I instead convert the list to a >> tuple and use element()? >> >> The lists are ~ 10 to 30 elements in length. (<-- not an attempt to define a >> frame). >> >> For curiosity, does the below tail-recurse correctly? >> >> listOrd(_, []) -> nil; >> >> listOrd(Searched, List) when is_list(List) -> >> >> listOrd(Searched, List, 1). >> >> listOrd(_, [], _) -> nil; >> >> listOrd(Searched, [ Element | Tail ], Count) -> >> >> case Searched == Element of >> true -> Count; >> _ -> listOrd(Searched, Tail, Count + 1) >> end. >> >> >> >> Thanks for your time, >> Henning >> >> > > From hd2010@REDACTED Thu May 27 22:19:42 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 22:19:42 +0200 Subject: [erlang-questions] list_element() In-Reply-To: References: <4BFE8224.3020407@eonblast.com> Message-ID: <4BFED3DE.8090206@eonblast.com> Yes, thanks a lot. Starts to look acceptable. Actually listOrd(Searched, [ Searched | _ ], Count) -> Count; listOrd(Searched, [_ | Tail ], Count) -> listOrd(Searched, Tail, Count+1). Henning Ariel Gomez wrote: > you do not need to use "if", use pattern matching. > > .. > .. > listOrd(Searched, [ Searched | Tail ], Count) -> > Count; > listOrd(Searched, [_ | Tail ], Count) -> > listOrd(Searched, Tail, Count+1). > > > 2010/5/27 Henning Diedrich > > > Hi list, > > I need to access a list element by index. > > I wrote a function but it looks most clumsy. > > I am looking for maximal performance. Should I instead convert the > list to a tuple and use element()? > > The lists are ~ 10 to 30 elements in length. (<-- not an attempt > to define a frame). > > For curiosity, does the below tail-recurse correctly? > > listOrd(_, []) -> nil; > > listOrd(Searched, List) when is_list(List) -> > > listOrd(Searched, List, 1). > > listOrd(_, [], _) -> nil; > > listOrd(Searched, [ Element | Tail ], Count) -> > > case Searched == Element of > true -> Count; > _ -> listOrd(Searched, Tail, Count + 1) > end. > > > > Thanks for your time, > Henning > > From hd2010@REDACTED Thu May 27 22:27:41 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 27 May 2010 22:27:41 +0200 Subject: [erlang-questions] list_element() In-Reply-To: References: <4BFE8224.3020407@eonblast.com> <4BFE8479.4010402@llaisdy.com> <4BFE869B.2060407@eonblast.com> Message-ID: <4BFED5BD.7050708@eonblast.com> I did a naive benchmark and it looks that foldl, maybe because of the fun() (?) is a lot slower? Even when looking up the last parameter in a list, when it is not aggravated by the fact that it traverses the whole list always. I thought that may be if interest. Maybe somebody knows the reason? 'if' replaced by pattern matching as you suggested, Ariel. 2> c(listord). 2> listord:run(). *listord1: run 70 wall 153 ms.* listord2: run 180 wall 377 ms. listord3: run 190 wall 416 ms. listord4: run 1240 wall 2181 ms. ok -module(listord). -export([run/0,listOrd3Folder/2]). run() -> List = [<<"Mabar">>, <<"Mibar">>, <<"Mobar">>, <<"Mubar">>, <<"Pabar">>, <<"Pebar">>, <<"Pibar">>, <<"Lubar">>, <<"Labar">>, <<"Lebar">>, <<"Libar">> ], N = 100000, S = <<"Libar">>, % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . statistics(runtime), statistics(wall_clock), for(1,N, fun() -> listOrd1(S, List) end), {_, RunTime1} = statistics(runtime), {_, WallClock1} = statistics(wall_clock), io:format("listord1: run ~p wall ~p ms.~n", [RunTime1, WallClock1]), % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . statistics(runtime), statistics(wall_clock), for(1,N, fun() -> listOrd2(S, List) end), {_, RunTime2} = statistics(runtime), {_, WallClock2} = statistics(wall_clock), io:format("listord2: run ~p wall ~p ms.~n", [RunTime2, WallClock2]), % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . statistics(runtime), statistics(wall_clock), for(1,N, fun() -> listOrd3(S, List) end), {_, RunTime3} = statistics(runtime), {_, WallClock3} = statistics(wall_clock), io:format("listord3: run ~p wall ~p ms.~n", [RunTime3, WallClock3]), % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . statistics(runtime), statistics(wall_clock), for(1,N, fun() -> listOrd4(S, List) end), {_, RunTime4} = statistics(runtime), {_, WallClock4} = statistics(wall_clock), io:format("listord4: run ~p wall ~p ms.~n", [RunTime4, WallClock4]). % . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . for(N,N,F) -> F(); for(I,N,F) -> F(), for(I+1,N,F). % ----------------------------------------------------------------------------- *listOrd1(Searched, List) when is_list(List) -> listOrd1(Searched, List, 1). listOrd1(_, [], _) -> nil; listOrd1(Searched, [ Searched | _ ], Count) -> Count; listOrd1(Searched, [ _ | Tail ], Count) -> listOrd1(Searched, Tail, Count + 1).* % ----------------------------------------------------------------------------- listOrd2(Searched, List) -> {_,I} = lists:foldl( fun(Item,{Acc,Index})-> if (Item==Searched) -> {Acc,Acc}; true-> {Acc+1,Index} end end, {1,nil}, List), I. % ----------------------------------------------------------------------------- listOrd3(Searched, List) -> {_,_,I} = lists:foldl( fun listord:listOrd3Folder/2, {Searched,1,nil}, List), I. listOrd3Folder(Item,{Searched,Acc,Index})-> if (Item==Searched) -> {Searched,Acc,Acc}; true-> {Searched,Acc+1,Index} end. % ----------------------------------------------------------------------------- listOrd4(Searched, List) -> L = length(List), hd([ Y || X <- List, Y <- lists:seq(1,L), X == Searched ]). Ariel Gomez wrote: > >lists:foldl(fun(Item,{Acc,Index})-> if(Item=="G")-> {Acc,Acc}; true-> > {Acc+1,Index} end end,{1,nil},["A","B","C","D","E","F","G","H","I","J" > ,"K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"]). > {26,7} > > 2010/5/27 Henning Diedrich > > > No, look the index up by the content. > > That's a misleading title. > > > Ivan Uemlianin wrote: > > Dear Henning > > Do you mean you need to access the nth element of a list? Is > lists:nth/2 no good? > > http://www.erlang.org/doc/man/lists.html#nth-2 > > Best wishes > > Ivan > > On 27/05/2010 15:31, Henning Diedrich wrote: > > Hi list, > > I need to access a list element by index. > > I wrote a function but it looks most clumsy. > > I am looking for maximal performance. Should I instead > convert the list to a tuple and use element()? > > The lists are ~ 10 to 30 elements in length. (<-- not an > attempt to define a frame). > > For curiosity, does the below tail-recurse correctly? > > listOrd(_, []) -> nil; > > listOrd(Searched, List) when is_list(List) -> > > listOrd(Searched, List, 1). > > listOrd(_, [], _) -> nil; > > listOrd(Searched, [ Element | Tail ], Count) -> > > case Searched == Element of > true -> Count; > _ -> listOrd(Searched, Tail, Count + 1) > end. > > > > Thanks for your time, > Henning > > > > > From ok@REDACTED Fri May 28 00:57:09 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 28 May 2010 10:57:09 +1200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> Message-ID: <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> I think it may be worth pointing out that this is one of the standard examples to show that there are things a language with smashable arrays can do faster than one without them. Testing is something Erlang programmers seem to believe it. Random testing is a very useful tool. It might be a good idea to move the core random number generator in random: into C (at present random:uniform() is quite slow) and perhaps even replace it by a better RNG. At the same time, it would be a good idea to provide random:list_permutation(List) -> List' random:tuple_permutation(Tuple) -> Tuple' as _fast_ building blocks. From ok@REDACTED Fri May 28 01:27:25 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 28 May 2010 11:27:25 +1200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFE6FB8.2040607@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> <4BFE6FB8.2040607@erix.ericsson.se> Message-ID: On May 28, 2010, at 1:12 AM, Bj?rn-Egil Dahlberg wrote: > Is there no need for assigning unique numbers to the list when > sorting it? I guess it boils down to how fair we want the list to be > unsorted (or how one wants to word it). If we really had something generating 53-bit random floats, the changes that a million-element list would get at least one duplicate would be 1 - (1 - 2**-53)**1,000,000 or roughly 2**-33. Or put it another way: stop worrying. The RNG that Erlang uses has worse problems than that. It was great for its purpose in Quintus Prolog >25 years ago, but it has long since been surpassed. Put another way, the problem of assigning unique numbers *is* the problem of generating random permutations (wearing a wig and a false nose). From hd2010@REDACTED Fri May 28 03:20:40 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 28 May 2010 03:20:40 +0200 Subject: [erlang-questions] The If expression In-Reply-To: References: <4BCF85D6.3070205@eonblast.com> <4BCFA1A5.9000900@eonblast.com> Message-ID: <4BFF1A68.7070505@eonblast.com> I think I found my one-armed-if at long last! Shame on everybody standing by who knew it all along and chuckled with glee: Index /= null orelse throw(index_null). *love it* *beautiful* *ouch* *ey! stop that!* I am still trying hard to avoid the need for a one armed if as per the reasons brought up in this thread. Many times I succeed and the extra effort pays off in a yet better understanding of the FP ways. However, whenever the desire is for more informative error messages --- over 'just let it crash' --- anything else clutters the code and one-armed ifs *are* the right thing to use there because I don't care for all the other cases. In such cases I am not interested in what value may be returned because I will break away anyway. And better error messages are a legitimate constraint sometimes. I understand it's not in Erlang's heritage in some way. Message passing and IO are other areas where side effects are the focus, not return values. And where therefore one-armed ifs could be a honest thing. From all the help, which I want to say *thank you* for, the following had been the strongest advice that stuck with me in its simplicity and fair abstraction: Robert Virding wrote: > ... however you choose to use 'if', or any other > construction for that matter, Erlang is a functional language and > everything returns, and must return, a value. Thanks, Robert, this reminder helped to get into the right thinking often times! > So even if 'if' didn't > need to have an explicit default or else case then it would still have > to return a value. We would never be able agree on a default value > anyway. :-) 'nil' is not a good value here as it has no inherent > meaning in Erlang as it does in Lisp. > It probably held true 80% of the times when I ran into a subjective (<- qualifier inserted for Rich!) need of a one-armed-if. But whenever exceptions come into play, the return value argument can become mute I found, in FP. As suggested by the fact that the result of the after-body of try-catch-after constructs are discarded as well. Currently I'd argue that in an analogous way the second arm of 'case' or 'if' is not necessarily-necessary when the only arm that matters is ending on throw/erlang:error/exit. If I wisen up, I promise I'll retract. I did find very many occasions where instead of a 'case', the correct pattern matching in the function head made all the difference, or a 'case' structure could with some deliberation be rewritten to have a sensical second arm. But especially when I need to throw an error /from/ a place 'higher up' in the call stack to put more information with it, I couldn't always find a sensible way to use the second arm of and 'if' or 'case'. (Re throwing from higher up: e.g. a list may have been consumed in its entirety 'lower down' when I realize a dead end is hit. And so I can't show it in the error message if it's thrown from there. Unless I add a parameter.) Thank you again for all your thoughts, they are very much appreciated. Henning From ok@REDACTED Fri May 28 07:16:03 2010 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 28 May 2010 17:16:03 +1200 Subject: [erlang-questions] The If expression In-Reply-To: <4BFF1A68.7070505@eonblast.com> References: <4BCF85D6.3070205@eonblast.com> <4BCFA1A5.9000900@eonblast.com> <4BFF1A68.7070505@eonblast.com> Message-ID: <0F792001-2332-419A-AC05-C6538D2315DF@cs.otago.ac.nz> On May 28, 2010, at 1:20 PM, Henning Diedrich wrote: > I think I found my one-armed-if at long last! > > Shame on everybody standing by who knew it all along and chuckled > with glee: > > Index /= null orelse throw(index_null). Sounds like you've rediscovered Perl's || die ; idiom. It's generally recommended in Erlang to use 'error' for errors, not 'throw'. It's not really clear to me how much of an improvement this is over if -> ok end > > But whenever exceptions come into play, the return value argument > can become mute I found, in FP. I think you mean "moot" (mOOt, subject to debate) not "mute" (mEWt, silent). > As suggested by the fact that the result of the after-body of try- > catch-after constructs are discarded as well. Currently I'd argue > that in an analogous way the second arm of 'case' or 'if' is not > necessarily-necessary when the only arm that matters is ending on > throw/erlang:error/exit. If I wisen up, I promise I'll retract. I don't think anybody was arguing against if -> end when the *intention* is that the process should crash/raise an exception on failure. Indeed, no second arm is required in this case. But then, the programmer's intentions are much more clearly stated if you write ?define(ASSERT(Guard), if Guard -> ok end). ... ?ASSERT(), > But especially when I need to throw an error /from/ a place 'higher > up' in the call stack to put more information with it, I couldn't > always find a sensible way to use the second arm of and 'if' or > 'case'. Give an example. With exception handling there are at least two separate issues: - unwind-protect; ensuring that actions are properly undone when an exception occurs - recovery and resumption. The big issue about exception handling is that the exception has to be *recognised*, *understood*, and *recovered from*. One would have to see the context, but things like 'index_null' have a nasty habit of referring to data other than stuff the original caller passed in, and might not even be able to name. It really is important to see a real example here. asso From huss01@REDACTED Fri May 28 08:42:53 2010 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Fri, 28 May 2010 08:42:53 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> <4BFE6FB8.2040607@erix.ericsson.se> Message-ID: 2010/5/28 Richard O'Keefe > > On May 28, 2010, at 1:12 AM, Bj?rn-Egil Dahlberg wrote: > >> Is there no need for assigning unique numbers to the list when sorting it? >> I guess it boils down to how fair we want the list to be unsorted (or how >> one wants to word it). >> > > If we really had something generating 53-bit random floats, > the changes that a million-element list would get at least one > duplicate would be > 1 - (1 - 2**-53)**1,000,000 > or roughly 2**-33. Or put it another way: stop worrying. The > RNG that Erlang uses has worse problems than that. It was > great for its purpose in Quintus Prolog >25 years ago, but it > has long since been surpassed. > Even though I used random:uniform/2 in the example I posted, any serious use of random numbers should use the RNG functions in the crypto module instead. For the "tag and sort", use crypto:rand_bytes/1 and you can choose how many random bits you want the tag to have (and thus the probability of collision). For instance, tagging with crypto:rand_bytes(7) will get you a 56 bit random binary with good quality. Regards. /H?kan From rtrlists@REDACTED Fri May 28 09:54:03 2010 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 28 May 2010 08:54:03 +0100 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4BFED171.1020100@eonblast.com> References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: On Thu, May 27, 2010 at 9:09 PM, Henning Diedrich wrote: > Thanks a lot James! > > am I right then to conclude that mutating something in a list will be the > faster the more the data is spread out deeper over several levels? > > > You can't mutate! So your examples don't really make sense. You can only build new structures. You cannot change them once their built. For mutation you need to look at ets, dets, mnesia, processes and messages. Robby From johanmon@REDACTED Fri May 28 10:15:38 2010 From: johanmon@REDACTED (Johan Montelius) Date: Fri, 28 May 2010 10:15:38 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4BFED171.1020100@eonblast.com> References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: On Thu, 27 May 2010 22:09:21 +0200, Henning Diedrich wrote: > A = { { B, C }, { D, E }, {F, G, H } } > because if I change e.g. B, only { B', C } will be created as new > internal list of pointers, as opposed to the whole of { B', C, D, E, F, > G, H }. Which should for larger lists and tuples, and frequent changes, > start to make a difference? Not exactly. Lets say we have A as above and want to construct a new tuple X = {{B', C}, {D, E}, {F, G, H }} What we then have to do is construct a new tuple with three elements X = { _ , _ , _} We then have to construct a new tuple for {B', C} X = {{_, _}, _, _} and then we of course have to construct B' X = {{B',_}, _,_} so we have constructed two tuples, one of size tree and one of size two (plus B'), all other structurs C, {D,E} and {F,G,H} are shared. If we instead have A = {B, C, D, E, F, G, H } and want to construct X = {B', C, D, E, F, G, H} we construct one new tuple X = {_, _, _ , _, _, _, _} and then B' X = {B', _, _ , _, _, _, _} so now we have only constructed one tuple of size seven. All other structures C,D,E,F,G,H are shared. The difference is in this case marginal (and I don't know which one executes faster). If we look at larger structures the difference become apparent. Assume you have a N elements in a set and want to update these element one by one. One way is to represent this a tuple of size N. Each update operation would then construct a new tuple of size N. All elements in the tuple would be shared but the new tuple has one element that differs. If we instead represent this as a list of length N we would have to construct a new list to the point where we have located the element that we want to change. Assume we want to change E: A = [B | [C | [D | [E | [F | [G | [H| []]]]]]]] X = [_ | [_ | [_ | [E' | _]]]] Depending in how far down E is found we have to construct new list cells for all element before E. In the example above this is four list cells [_|_]. In average thins means N/2 new list cells for each update. This is for large N more efficient than having to construct a new tuple of size N. A even better way, if N is large, is to construct a tree. A = {tree, {tree, {tree, {leaf, B}, {leaf, C}}, {tree, {leaf, D}, {leaf, E}}} {tree, {tree, {leaf, F}, {leaf, G}}, {leaf, H}}} Now if we want to change E then we have A = {tree, {tree, _, {tree, _, {leaf, E'}}} _} So we construct new tree tuples all the way down to the location of E. All other structures are shared. In general this means that we will construct lg(N) new tree tuples. So a tuple of size N, N/2 new list cells or lg(N) new tree tuples. The data structures of course have other implications when it comes to reading and adding elements. Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From gustav.simonsson@REDACTED Fri May 28 10:46:38 2010 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Fri, 28 May 2010 08:46:38 +0000 (GMT) Subject: [erlang-questions] Problem with io:fwrite/2 ? In-Reply-To: <1382244876.406381275036345789.JavaMail.root@zimbra> Message-ID: <952834456.406471275036398277.JavaMail.root@zimbra> Hard to say without knowing how your CL#'CallLog'.file implements the Erlang IO protocols, does it always reply on the form {io_reply, ReplyAs, Reply}? Thing is, the main receive clause in wait_io_mon_reply(From, Mref) in stdlib/io.erl waits for either a io_reply or an exit/down message, so it could potentially hang there if your CL#'CallLog'.file is not correctly implemented. BR, Gustav Simonsson ----- Original Message ----- From: "Steven Gardell" To: erlang-questions@REDACTED Sent: Thursday, 27 May, 2010 19:01:02 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Problem with io:fwrite/2 ? Running R13B04 under Linux RES 4. It seems that io:fwrite/2 is hanging "most of the time." With the following code segment: io:format("BEFORE FULL WRITE~n"), io:fwrite(CL#'CallLog'.file, "Calls: ~w Rate: ~w Trans: ~w %SlowAdd: ~.1f %SlowMod: ~.1f %SlowOther ~.1f %SlowTotal ~.1f %Slow2: ~.1f %Slow3: ~.1f Failed ~w Unexpected: ~w~n", [CL#'CallLog'.callCount, Rate, CL#'CallLog'.transCount, SlowAdd1, SlowMod1, SlowOther1, SlowTotal1, SlowTotal2, SlowTotal3, CL#'CallLog'.failedCount, CL#'CallLog'.unexpectedMsg]), io:format("BEFORE EMPTY LINE~n"), io:fwrite(CL#'CallLog'.file,"~n"), io:format("AFTER EMPTY LINE~n"). I see: 1> LOADTEST: Ready loadtest:onecall(twoterm4trans). Test: 5/27/2010 12:47:36 Rate: 1 Hold: 1 Calls:1 {go,1,1,0,1,1,twoterm4trans} BEFORE FULL WRITE BEFORE EMPTY LINE 2> The subsequent behavior of the program leads me to believe we never returned from fwrite(). From sgardell@REDACTED Fri May 28 11:23:22 2010 From: sgardell@REDACTED (Gardell, Steven) Date: Fri, 28 May 2010 05:23:22 -0400 Subject: [erlang-questions] Problem with io:fwrite/2 ? In-Reply-To: <952834456.406471275036398277.JavaMail.root@zimbra> References: <1382244876.406381275036345789.JavaMail.root@zimbra> <952834456.406471275036398277.JavaMail.root@zimbra> Message-ID: <6B3E29721F78364AA2C43697AFB15D910215280B@sonusmail07.sonusnet.com> Thanks. Being somewhat new to Erlang, I maybe didn't provide quite enough info. I am using the standard file IO: call_logger()-> {ok,File} = file:open("loadlog.txt", [append]), call_logger(File). call_logger(File)-> call_collector(#'CallLog'{file=File}), call_logger(File). Note that that this works just fine for fwrite/3. -----Original Message----- From: Gustav Simonsson [mailto:gustav.simonsson@REDACTED] Sent: Friday, May 28, 2010 4:47 AM To: Gardell, Steven Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Problem with io:fwrite/2 ? Hard to say without knowing how your CL#'CallLog'.file implements the Erlang IO protocols, does it always reply on the form {io_reply, ReplyAs, Reply}? Thing is, the main receive clause in wait_io_mon_reply(From, Mref) in stdlib/io.erl waits for either a io_reply or an exit/down message, so it could potentially hang there if your CL#'CallLog'.file is not correctly implemented. BR, Gustav Simonsson ----- Original Message ----- From: "Steven Gardell" To: erlang-questions@REDACTED Sent: Thursday, 27 May, 2010 19:01:02 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Problem with io:fwrite/2 ? Running R13B04 under Linux RES 4. It seems that io:fwrite/2 is hanging "most of the time." With the following code segment: io:format("BEFORE FULL WRITE~n"), io:fwrite(CL#'CallLog'.file, "Calls: ~w Rate: ~w Trans: ~w %SlowAdd: ~.1f %SlowMod: ~.1f %SlowOther ~.1f %SlowTotal ~.1f %Slow2: ~.1f %Slow3: ~.1f Failed ~w Unexpected: ~w~n", [CL#'CallLog'.callCount, Rate, CL#'CallLog'.transCount, SlowAdd1, SlowMod1, SlowOther1, SlowTotal1, SlowTotal2, SlowTotal3, CL#'CallLog'.failedCount, CL#'CallLog'.unexpectedMsg]), io:format("BEFORE EMPTY LINE~n"), io:fwrite(CL#'CallLog'.file,"~n"), io:format("AFTER EMPTY LINE~n"). I see: 1> LOADTEST: Ready loadtest:onecall(twoterm4trans). Test: 5/27/2010 12:47:36 Rate: 1 Hold: 1 Calls:1 {go,1,1,0,1,1,twoterm4trans} BEFORE FULL WRITE BEFORE EMPTY LINE 2> The subsequent behavior of the program leads me to believe we never returned from fwrite(). From john.hughes@REDACTED Fri May 28 12:06:55 2010 From: john.hughes@REDACTED (John Hughes) Date: Fri, 28 May 2010 12:06:55 +0200 Subject: [erlang-questions] The If expression In-Reply-To: <4BFF1A68.7070505@eonblast.com> References: <4BCF85D6.3070205@eonblast.com> <4BCFA1A5.9000900@eonblast.com> <4BFF1A68.7070505@eonblast.com> Message-ID: I use a list comprehension when I need a one armed if with a side effect. [io:format("Values: ~p\n",[Vs]) || Vs/=[]] Looks funny at first but you soon get used to it. John Sent from my iPhone On 28 May 2010, at 03:20, Henning Diedrich wrote: > I think I found my one-armed-if at long last! > > Shame on everybody standing by who knew it all along and chuckled > with glee: > > Index /= null orelse throw(index_null). > > > *love it* *beautiful* *ouch* *ey! stop that!* > > I am still trying hard to avoid the need for a one armed if as per > the reasons brought up in this thread. > > Many times I succeed and the extra effort pays off in a yet better > understanding of the FP ways. > > However, whenever the desire is for more informative error messages > --- over 'just let it crash' --- anything else clutters the code and > one-armed ifs *are* the right thing to use there because I don't > care for all the other cases. In such cases I am not interested in > what value may be returned because I will break away anyway. > > And better error messages are a legitimate constraint sometimes. I > understand it's not in Erlang's heritage in some way. > > Message passing and IO are other areas where side effects are the > focus, not return values. And where therefore one-armed ifs could be > a honest thing. > > From all the help, which I want to say *thank you* for, the > following had been the strongest advice that stuck with me in its > simplicity and fair abstraction: > > Robert Virding wrote: >> ... however you choose to use 'if', or any other >> construction for that matter, Erlang is a functional language and >> everything returns, and must return, a value. > Thanks, Robert, this reminder helped to get into the right thinking > often times! >> So even if 'if' didn't >> need to have an explicit default or else case then it would still >> have >> to return a value. We would never be able agree on a default value >> anyway. :-) 'nil' is not a good value here as it has no inherent >> meaning in Erlang as it does in Lisp. >> > It probably held true 80% of the times when I ran into a subjective > (<- qualifier inserted for Rich!) need of a one-armed-if. > > But whenever exceptions come into play, the return value argument > can become mute I found, in FP. As suggested by the fact that the > result of the after-body of try-catch-after constructs are discarded > as well. Currently I'd argue that in an analogous way the second arm > of 'case' or 'if' is not necessarily-necessary when the only arm > that matters is ending on throw/erlang:error/exit. If I wisen up, I > promise I'll retract. > > I did find very many occasions where instead of a 'case', the > correct pattern matching in the function head made all the > difference, or a 'case' structure could with some deliberation be > rewritten to have a sensical second arm. > > But especially when I need to throw an error /from/ a place 'higher > up' in the call stack to put more information with it, I couldn't > always find a sensible way to use the second arm of and 'if' or > 'case'. (Re throwing from higher up: e.g. a list may have been > consumed in its entirety 'lower down' when I realize a dead end is > hit. And so I can't show it in the error message if it's thrown from > there. Unless I add a parameter.) > > Thank you again for all your thoughts, they are very much appreciated. > > Henning From bflatmaj7th@REDACTED Fri May 28 12:46:50 2010 From: bflatmaj7th@REDACTED (Richard Andrews) Date: Fri, 28 May 2010 20:46:50 +1000 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: On Fri, May 28, 2010 at 6:15 PM, Johan Montelius wrote: > If we instead have > > A = {B, C, D, E, F, G, H } > > and want to construct > > X = {B', C, D, E, F, G, H} > > we construct one new tuple > > X = {_, _, _ , _, _, _, _} > > and then B' > > X = {B', _, _ , _, _, _, _} > > so now we have only constructed one tuple of size seven. All other > structures C,D,E,F,G,H are shared. but... IIRC, if A is not used in the code after X is bound, the erlang compiler can optimise this to an in-place operation where the storage for A is reused for X. So only the B element pointer is changed to point to B' and no allocation is required. Is this not one of the reasons for record syntax? eg. A = #myRecord{B,C,D,E,F,G,H}, X = A#myRecord{B2}, From kenneth.lundin@REDACTED Fri May 28 13:38:38 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 28 May 2010 13:38:38 +0200 Subject: [erlang-questions] ERL_FLOAT_EXT In-Reply-To: <4BFE4710.70107@aleynikov.org> References: <4BFE4710.70107@aleynikov.org> Message-ID: Hi Serge, I recommend you to contribute the patch according to the instructions on GitHub http://wiki.github.com/erlang/otp/submitting-patches and to also provide new test cases, then it is very likely that we will include it in the distribution. Regarding the compatibility we don't think it is an issue any more. If new versions of ei/erl_interface always use the ERL_FLOAT_EXT format will be ok. The number of contributions is continually increasing and it is much easier for us if we can handle all contributions in the same way. This is also beneficial for the overall quality /Kenneth Erlang/OTP Ericsson. On Thu, May 27, 2010 at 12:18 PM, Serge Aleynikov wrote: > A while ago I submitted a patch for ei to support IEEE 754 double > encoding/decoding compliant with ERL_FLOAT_EXT format. > > http://www.erlang.org/cgi-bin/ezmlm-cgi/3/256 > > Is it possible for it to get included in the distribution? ?The rationale is > that the emulator supports compact encoding of doubles in external binary > format, but ei doesn't. > > Serge > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From torben.lehoff@REDACTED Fri May 28 13:51:52 2010 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 28 May 2010 13:51:52 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <3F9829DB-AAE9-4398-8AE3-6A3E292B8202@cs.otago.ac.nz> <4BFE6FB8.2040607@erix.ericsson.se> Message-ID: (Re-send to list due to mail address change on gmail) On Fri, May 28, 2010 at 08:42, H?kan Huss wrote: > 2010/5/28 Richard O'Keefe > > > > > On May 28, 2010, at 1:12 AM, Bj?rn-Egil Dahlberg wrote: > > > >> Is there no need for assigning unique numbers to the list when sorting > it? > >> I guess it boils down to how fair we want the list to be unsorted (or > how > >> one wants to word it). > >> > > > > If we really had something generating 53-bit random floats, > > the changes that a million-element list would get at least one > > duplicate would be > > 1 - (1 - 2**-53)**1,000,000 > > or roughly 2**-33. Or put it another way: stop worrying. The > > RNG that Erlang uses has worse problems than that. It was > > great for its purpose in Quintus Prolog >25 years ago, but it > > has long since been surpassed. > > > > Even though I used random:uniform/2 in the example I posted, any serious > use > of random numbers should use the RNG functions in the crypto module > instead. > For the "tag and sort", use crypto:rand_bytes/1 and you can choose how many > random bits you want the tag to have (and thus the probability of > collision). For instance, tagging with crypto:rand_bytes(7) will get you a > 56 bit random binary with good quality. > > Regards. > /H?kan > Ah, you just happened to touch one of my pet peeves here! The crypto module is great - for server installations. If you want to distribute an application that needs crypto functionality you have to figure out how to package the OpenSSL library as well (just correct me if I am wrong). To surpass that problem I have created a RNG based on a SHA function including the proper seeding - see http://github.com/lehoff/cryptographicfor some of the initial code. WARNING: THAT CODE IS WORK IN PROGRESS, but the RNG seems to work. The seeding is the only thing that requires some C code, but that is so little that it is easy to write it up for the most popular platforms and then add new platforms in case the application becomes popular. So the code is in Erlang plus a C port - that seems easier to ship than the crypto application + OpenSSL, but I could be mistaken. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From rvirding@REDACTED Fri May 28 13:56:55 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 28 May 2010 13:56:55 +0200 Subject: [erlang-questions] Problem with io:fwrite/2 ? In-Reply-To: <6B3E29721F78364AA2C43697AFB15D9102D94F7C@sonusmail07.sonusnet.com> References: <6B3E29721F78364AA2C43697AFB15D9102D94F7C@sonusmail07.sonusnet.com> Message-ID: Some points here: - There is no difference at all between io:fwrite and io:format. Format is the older name, but I added the name fwrite to match fread. - In both cases format/2 and fwrite/2 the arguments are fwrite(Format, Data), if you want to send it to a specific iodevice then you need to use the 3 argument calls. The different versions are: fwrite(Format) %Format must not need data fwrite(Format, Data) fwrite(IoDevice, Format, Data) Same for format of course. Robert On 27 May 2010 19:01, Gardell, Steven wrote: > Running R13B04 under Linux RES 4. > It seems that io:fwrite/2 is hanging "most of the time." With the > following code segment: > > ?io:format("BEFORE FULL WRITE~n"), > ? ?io:fwrite(CL#'CallLog'.file, > ? ? ? ?"Calls: ~w Rate: ~w Trans: ~w %SlowAdd: ~.1f %SlowMod: ~.1f > %SlowOther ~.1f %SlowTotal ~.1f %Slow2: ~.1f %Slow3: ~.1f Failed ~w > Unexpected: ~w~n", > ? ? ? ?[CL#'CallLog'.callCount, > ? ? ? ?Rate, > ? ? ? ?CL#'CallLog'.transCount, > ? ? ? ?SlowAdd1, > ? ? ? ?SlowMod1, > ? ? ? ?SlowOther1, > ? ? ? ?SlowTotal1, > ? ? ? ?SlowTotal2, > ? ? ? ?SlowTotal3, > ? ? ? ?CL#'CallLog'.failedCount, > ? ? ? ?CL#'CallLog'.unexpectedMsg]), > > ? ?io:format("BEFORE EMPTY LINE~n"), > ? ?io:fwrite(CL#'CallLog'.file,"~n"), > ? ?io:format("AFTER EMPTY LINE~n"). > > I see: > > 1> LOADTEST: Ready > loadtest:onecall(twoterm4trans). > Test: 5/27/2010 12:47:36 Rate: 1 Hold: 1 Calls:1 > {go,1,1,0,1,1,twoterm4trans} > BEFORE FULL WRITE > BEFORE EMPTY LINE > 2> > > The subsequent behavior of the program leads me to believe > we never returned from fwrite(). > From johanmon@REDACTED Fri May 28 15:09:11 2010 From: johanmon@REDACTED (Johan Montelius) Date: Fri, 28 May 2010 15:09:11 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: On Fri, 28 May 2010 12:46:50 +0200, Richard Andrews wrote: > IIRC, if A is not used in the code after X is bound, the erlang > compiler can optimise this to an in-place operation where the storage > for A is reused for X. So only the B element pointer is changed to > point to B' and no allocation is required. Is this not one of the > reasons for record syntax? I would be happily surprised if that is the case. Determining if A is the last reference to the structure involves either a lot of program analysis or run-time reference counting. The record syntax solves a problem of readability, maintainability etc of src code. To see why the compiler will have a hard time identifying when it is safe to do destructive updates look at the following: foo(X) -> case X of {A, B, C} -> B1 = mod(B), Y = {A, mod(B), C} zot(Y) end. If X is the last reference to the structure {A, B, C} we can replace B with B1 and just hand it to zot/1. The problem is of course that we don't know if X is used somewhere else. If this was the call to foo/1 : foo(X), bar(X), : then we would make a mistake in destroying the structure. To figure out at compile time when it is safe to reclaim structures is tricky. Johan -- Associate Professor Johan Montelius Royal Institute of Technology - KTH School of Information and Communication Technology - ICT From serge@REDACTED Fri May 28 15:29:04 2010 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 28 May 2010 09:29:04 -0400 Subject: [erlang-questions] ERL_FLOAT_EXT In-Reply-To: References: <4BFE4710.70107@aleynikov.org> Message-ID: <4BFFC520.3080606@aleynikov.org> Thanks Kenneth! I'll use the instructions for future patch submissions. Steve Vinoski kindly offered his help to push this patch to github. I am a little unclear on your note below - did you mean to say that new versions of ei can use NEW_FLOAT_EXT introduced in the patch, or that they should continue using ERL_FLOAT_EXT? I assume it's the former. Serge On 5/28/2010 7:38 AM, Kenneth Lundin wrote: > Hi Serge, > > I recommend you to contribute the patch according to the instructions on GitHub > http://wiki.github.com/erlang/otp/submitting-patches > > and to also provide new test cases, then it is very likely that we > will include it in the distribution. > Regarding the compatibility we don't think it is an issue any more. If > new versions of ei/erl_interface > always use the ERL_FLOAT_EXT format will be ok. > > The number of contributions is continually increasing and it is much > easier for us if we > can handle all contributions in the same way. This is also beneficial > for the overall quality > > /Kenneth Erlang/OTP Ericsson. > > On Thu, May 27, 2010 at 12:18 PM, Serge Aleynikov wrote: >> A while ago I submitted a patch for ei to support IEEE 754 double >> encoding/decoding compliant with ERL_FLOAT_EXT format. >> >> http://www.erlang.org/cgi-bin/ezmlm-cgi/3/256 >> >> Is it possible for it to get included in the distribution? The rationale is >> that the emulator supports compact encoding of doubles in external binary >> format, but ei doesn't. >> >> Serge >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From rvirding@REDACTED Fri May 28 18:00:40 2010 From: rvirding@REDACTED (Robert Virding) Date: Fri, 28 May 2010 18:00:40 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4BFED171.1020100@eonblast.com> References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: As others have pointed out data in Erlang is defined to immutable. This implies that if you wish to do destructive operations then you have to be sure they are not seen anywhere. In general this can be quite difficult as it is often not trivial to see if a data structure, or a portion of it, is shared. The compiler is very conservative about this and IIRC only does it when there is a sequence of setelement/3 where the temporaries are not exported and nothing is done between them. You typically get this when you do a single multiple record update, for example A1 = A0#foo{bar=57,baz="Robert",zip=now()} It is better if you can combine record updates like this instead of doing them one-by-one. I think it is easiest to view a complex data structure as a tree with generally different sized nodes. If you modify one of the nodes then the only other nodes you will have to rewrite are those nodes between the root of the tree and the modified node. Unless of course you are doing something special along the way. Or you have botched your code. :-) What sized nodes depends entirely upon what you are doing, the only general hint I can give is use that which fits best into your application. I mean it, really. Don't go complicating matters by picking weirdo data structures. So if you want a dynamic sequence then a list is usually the best, though not always. If you have 7 things which go together then put them in one 7 element tuple or in a record. I would only really start worrying about splitting my tuples if they get really big. Or if you feel that grouping them is what feels best. The only time I would seriously start thinking about what size thing to use is if you are writing more general library type packages. Generally having smaller nodes but a deeper resulting tree will lead to less rewriting, but as the tree is deeper it will cost more to traverse it and though the rewrites will be smaller there will be more of them. The opposite for wider flatter trees. It also depends on how you select which subnode to go to, this is, of course, very algorithm dependent. So in the array module each node contains 10 slots, IIRC, but here it is easy to select the right one using its index. Dicts also use indices and are wider. For more general search trees it becomes more complex and binary trees are popular, but I would say that it is more the algorithms which decide this. A list is really a degenerate binary tree. It all depends. This is clearly optimization and I would read http://en.wikipedia.org/wiki/Program_optimization "When to optimize" before getting to deep into this. Robert On 27 May 2010 22:09, Henning Diedrich wrote: > Thanks a lot James! > > am I right then to conclude that mutating something in a list will be the > faster the more the data is spread out deeper over several levels? > > A = { B, C, D, E, F, G, H } > > will be slower to change than, say > > A = { { B, C }, { D, E }, ?{F, G, H } } > > because if I change e.g. B, only { B', C } will be created as new ?internal > list of pointers, as opposed to the whole of { B', C, D, E, F, G, H }. Which > should for larger lists and tuples, and frequent changes, start to make a > difference? > > So this would be the fastes option? > > A = { { B }, { C }, { D }, { E }, ?{ F }, { G }, { H } } > > And if the tuple may be expected to change in size (elements not only > changeing but being added/deleted), this: > > A = { { { { B }, { C } }, { { D }, { E } } }, { { { F }, { G } }, { { H } } > } } > > Maybe I am wrong that there may be a difference between tuples and lists, > but is it ?the same for > > A = [ B, C, D, E, F, G, H ] > > slower to change than > > A = [ { B, C }, { D, E }, ?{ F, G, H } ] > > ? Or are list-elements changed faster, in general, virtually 'in-place' as > only one pointer is replaced? > > Hardly I guess, because the list itself must not be changed? So the whole > new list must be copied anyway, just like a tuple. > > Thanks for your take, > Henning > > > James Hague wrote: >> >> If you ignore message passing, then the copying/sharing issues in Erlang >> are >> just about the same as in Lisp and Scheme, so you may find some >> information >> that way. >> >> The short version is that some values are represented directly in a memory >> "cell," like atoms and (most) integers. Others are pointers to data stored >> elsewhere (list elements, or conses, which are two consecutive cells in >> memory) or tuples (N consecutive cells plus one for the size). Just by >> looking at code you can see where conses and tuples are created. If you >> use >> an existing "pointer" to data, then that data is not copied. >> >> Examples: >> >> B = {A, A, A} >> This creates a three element tuple (four memory cells). It doesn't matter >> if >> A is a list of a million elements or a tuple or the number 2. B just >> contains three cell-sized values (plus the size of the tuple). >> >> X = [57|List] >> This creates a single cons (2 cells). The first element is 57. The second >> is >> a pointer to List. So even though you've got two lists (X and List), all >> of >> the data is shared except for the first cons of X. >> >> > > From kenneth.lundin@REDACTED Fri May 28 20:18:58 2010 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Fri, 28 May 2010 20:18:58 +0200 Subject: [erlang-questions] ERL_FLOAT_EXT In-Reply-To: <4BFFC520.3080606@aleynikov.org> References: <4BFE4710.70107@aleynikov.org> <4BFFC520.3080606@aleynikov.org> Message-ID: I meant that new versions of ei can use NEW_FLOAT_EXT and don't need to support the ERL_FLOAT_EXT format at all. /Kenneth On Fri, May 28, 2010 at 3:29 PM, Serge Aleynikov wrote: > Thanks Kenneth! ?I'll use the instructions for future patch submissions. > Steve Vinoski kindly offered his help to push this patch to github. > > I am a little unclear on your note below - did you mean to say that new > versions of ei can use NEW_FLOAT_EXT introduced in the patch, or that they > should continue using ERL_FLOAT_EXT? ?I assume it's the former. > > Serge > > On 5/28/2010 7:38 AM, Kenneth Lundin wrote: >> >> Hi Serge, >> >> I recommend you to contribute the patch according to the instructions on >> GitHub >> http://wiki.github.com/erlang/otp/submitting-patches >> >> and to also provide new test cases, then it is very likely that we >> will include it in the distribution. >> Regarding the compatibility we don't think it is an issue any more. If >> new versions of ei/erl_interface >> always use the ERL_FLOAT_EXT format will be ok. >> >> The number of contributions is continually increasing and it is much >> easier for us if we >> can handle all contributions in the same way. This is also beneficial >> for the overall quality >> >> /Kenneth Erlang/OTP Ericsson. >> >> On Thu, May 27, 2010 at 12:18 PM, Serge Aleynikov >> ?wrote: >>> >>> A while ago I submitted a patch for ei to support IEEE 754 double >>> encoding/decoding compliant with ERL_FLOAT_EXT format. >>> >>> http://www.erlang.org/cgi-bin/ezmlm-cgi/3/256 >>> >>> Is it possible for it to get included in the distribution? ?The rationale >>> is >>> that the emulator supports compact encoding of doubles in external binary >>> format, but ei doesn't. >>> >>> Serge >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > From rennie@REDACTED Fri May 28 20:20:49 2010 From: rennie@REDACTED (=?UTF-8?B?SmVmZnJleSBSZW5uaWUgKOODrOODi+ODvO+8iQ==?=) Date: Fri, 28 May 2010 11:20:49 -0700 Subject: Freezing a process Message-ID: I've scoured the manuals unsuccessfully, and I've been unable to conjure the right Google query to answer the following question. hibernate() seems to get my half way there, but... Given a pid, can I freeze the process, serialize its stack and heap, send it down the wire and effectively resume it on another node? From torben.lehoff@REDACTED Fri May 28 23:13:17 2010 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 28 May 2010 23:13:17 +0200 Subject: [erlang-questions] Freezing a process In-Reply-To: References: Message-ID: On Fri, May 28, 2010 at 20:20, Jeffrey Rennie (???) wrote: > I've scoured the manuals unsuccessfully, and I've been unable to > conjure the right Google query to answer the following question. > hibernate() seems to get my half way there, but... > > Given a pid, can I freeze the process, serialize its stack and heap, > send it down the wire and effectively resume it on another node? > > Hi Jeffrey. I cannot give you an answer of how to do it in pure Erlang, but if you are using OTP you can take the internal state of the component out and send it to another machine. Apart from this function to get the state out you will need a start function that will restore the saved state from the old process. The OTP framework has the ability to provide takeover functionality for applicaitons, but if my memory serves me correct it involves a restart and is in nature more of a resilience feature than a mobility one. Depending on the details of your requirements there are many subtle things to consider in this, e.g., who keeps track of where a process with a given role is located (global could be the answer) and if the resume does not work how should the behaviour be? Ask one of the Ericsson guys and you will probably get an almost infinite list. The good news is that the tools to get it done is there - the bad news is that you have to put them together to meet your specific needs (this could also be good news since it means that there is a job to do!) Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From brady.mccary@REDACTED Sat May 29 00:09:02 2010 From: brady.mccary@REDACTED (Brady McCary) Date: Fri, 28 May 2010 17:09:02 -0500 Subject: Leex Character Class Complement Message-ID: erlang-questions, I would like to have the complement of a character class in a leex file, or to be able to subtract character classes. An example xrl file is as follows: %%% ---- Beginning of test.xrl ---- Definitions. A = [ab]{-}[b] Rules. {A}+ : {token, {a, TokenLine, TokenChars}}. Erlang code. % Nothing %%% ---- End of test.xrl ---- Compiling test.xrl and using is as follows: 1> leex:file(test,[{verbose, true}]), c(test), test:string("a"). Parsing file ./test.xrl, contained 1 rules. NFA contains 9 states, DFA contains 6 states, minimised to 6 states. Writing file ./test.erl, ok {error,{1,test,{illegal,"a"}},1} In the above test.xrl, I would hope that A = [a]. and that the rule {A}+ would match the string "a", but it is not so. Does anybody know how to do this? The reference for the analogous flex implementation of the subtraction operator is (search for {-} on the page): http://flex.sourceforge.net/manual/Patterns.html#Patterns Brady From rennie@REDACTED Sat May 29 00:18:12 2010 From: rennie@REDACTED (=?UTF-8?B?SmVmZnJleSBSZW5uaWUgKOODrOODi+ODvCk=?=) Date: Fri, 28 May 2010 15:18:12 -0700 Subject: [erlang-questions] Freezing a process In-Reply-To: References: Message-ID: On Fri, May 28, 2010 at 2:13 PM, Torben Hoffmann wrote: > > > On Fri, May 28, 2010 at 20:20, Jeffrey Rennie (???) > wrote: >> >> I've scoured the manuals unsuccessfully, and I've been unable to >> conjure the right Google query to answer the following question. >> hibernate() seems to get my half way there, but... >> >> Given a pid, can I freeze the process, serialize its stack and heap, >> send it down the wire and effectively resume it on another node? >> > > Hi Jeffrey. > > I cannot give you an answer of how to do it in pure Erlang, but if you are > using OTP you can take the internal state of the component out and send it > to another machine. > Apart from this function to get the state out you will need a start function > that will restore the saved state from the old process. > > The OTP framework has the ability to provide takeover functionality for > applicaitons, but if my memory serves me correct it involves a restart and > is in nature more of a resilience feature than a mobility one. > > Depending on the details of your requirements there are many subtle things > to consider in this, e.g., who keeps track of where a process with a given > role is located (global could be the answer) and if the resume does not work > how should the behaviour be? > Ask one of the Ericsson guys and you will probably get an almost infinite > list. > > The good news is that the tools to get it done is there - the bad news is > that you have to put them together to meet your specific needs (this could > also be good news since it means that there is a job to do!) Thanks for the analysis. I don't want to create work for anyone, as I don't have requirements yet. In other light-weight concurrency solutions I've seen, such as stackless Python, it's straight-forward to serialize processes. I was afraid I had overlooked something in Erlang. From rvirding@REDACTED Sat May 29 04:04:10 2010 From: rvirding@REDACTED (Robert Virding) Date: Sat, 29 May 2010 04:04:10 +0200 Subject: [erlang-questions] Leex Character Class Complement In-Reply-To: References: Message-ID: No, there is no way to do this in leex today and it is not something I have in the pipeline. Robert On 29 May 2010 00:09, Brady McCary wrote: > erlang-questions, > > I would like to have the complement of a character class in a leex > file, or to be able to subtract character classes. An example xrl file > is as follows: > > %%% ---- Beginning of test.xrl ---- > > Definitions. > > A = [ab]{-}[b] > > Rules. > > {A}+ : {token, {a, TokenLine, TokenChars}}. > > Erlang code. > > % Nothing > > %%% ---- End of test.xrl ---- > > Compiling test.xrl and using is as follows: > > 1> leex:file(test,[{verbose, true}]), c(test), test:string("a"). > Parsing file ./test.xrl, contained 1 rules. > NFA contains 9 states, DFA contains 6 states, minimised to 6 states. > Writing file ./test.erl, ok > {error,{1,test,{illegal,"a"}},1} > > In the above test.xrl, I would hope that A = [a]. and that the rule > {A}+ would match the string "a", but it is not so. Does anybody know > how to do this? The reference for the analogous flex implementation of > the subtraction operator is (search for {-} on the page): > > http://flex.sourceforge.net/manual/Patterns.html#Patterns > > Brady > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From hd2010@REDACTED Sat May 29 04:50:32 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 29 May 2010 04:50:32 +0200 Subject: Erlvolt Erlang VoltDB API In-Reply-To: <9A8C1395-1243-41C6-9EDA-C1460E20D9BD@gmx.net> References: <9A8C1395-1243-41C6-9EDA-C1460E20D9BD@gmx.net> Message-ID: <4C0080F8.6040002@eonblast.com> Hi Harald, please try http://svnmirror.voltdb.com/clientapi/erlang/trunk (lower case) Regards, Henning Harald Weppner wrote: > Hi Henning, > > is anonymous read-only access set up? When using >> svn checkout http://svnmirror.voltdb.com/ClientAPI/erlang/trunk >> > I am getting a "svn: Could not open the requested SVN filesystem" > > Thanks & cheerio, Harry. From ulf.wiger@REDACTED Sat May 29 05:51:38 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 29 May 2010 05:51:38 +0200 Subject: [erlang-questions] Freezing a process In-Reply-To: References: Message-ID: <4C008F4A.1080800@erlang-solutions.com> On 05/29/2010 12:18 AM, Jeffrey Rennie (???) wrote: > > In other light-weight concurrency solutions I've seen, such as > stackless Python, it's straight-forward to serialize processes. I was > afraid I had overlooked something in Erlang. In Termite (http://code.google.com/p/termite/), which is a Scheme dialect inspired by Erlang, it is also possible to migrate a process. In Erlang, it isn't possible to do it in a generic way, and transparently. There are a few details that get in the way, such as that the owner of a port must be a local process, and ports are by design not possible to move transparently. This is actually more or less an original design philosophy of Erlang. It was designed to support distribution transparency, such that it shouldn't matter where the other processes are, that are talking to you. At the same time, it had to be possible to be very explicit about where you spawn a process, since, if a process controls a hardware device, it should be running in a place where this is at all possible to do. Another issue is that new failure conditions arise when crossing a node boundary (such as disrupted communication and partial failure), and it is sometimes necessary to take this into account in your program. These things are not necessarily anathema to allowing the migration of processes. One could imagine a process flag forbidding the migration, for cases where the programmer thinks that code would break. But the OTP design philosophy is rather to move whole applications and being rather explicit about node boundaries, such that an OTP application instance always belongs to a specific node. OTOH, if a process is designed to be easily movable, in that it has no tricky local dependencies, it is trivial to write a migrate() function for it. If it is a behaviour, the sys module provides the means to suspend a process, extract its state, and resume it. But in the case of migration, you would rather start the process in its new location with the migrated state, and e.g. use the gen_server:enter_loop() function to resume it. BR, Ulf W --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From allanwegan@REDACTED Sat May 29 07:19:47 2010 From: allanwegan@REDACTED (Allan Wegan) Date: Sat, 29 May 2010 07:19:47 +0200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <4BFD16EF.6060206@erix.ericsson.se> References: <4BFD16EF.6060206@erix.ericsson.se> Message-ID: <4C00A3F3.9090100@allanwegan.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > However, there are some constraints on this function. It needs to be > generic enough, it needs to be fast enough and consume as little memory > as possible. Three constraints that seldom spell success when combined. > > My suggestion is included in this mail, but my question is: does anyone > have a faster, more generic and less memory consuming shuffle function? In an ideal world the following function would be realy fast (because of O(length(List) of the compiler optimized list_rearrange/4: % Call example: list_rearrange(fun randomizer/1, List) - -export([ list_rearrange/2, randomizer/1 ]). list_rearrange(_F, [] = L) -> L; list_rearrange(_F, [_] = L) -> L; list_rearrange(F, List) -> Input = list_to_tuple(List), Length = tuple_size(Input), Output = erlang:make_tuple(Length, 'undefined'), list_rearrange(F, Input, Length, Output) . list_rearrange(_F, _Input, 0 = _Length, Output) -> tuple_to_list(Output); list_rearrange( F, Input, Length, Output) -> {F_2, Chosen_pos} = F(Length), Chosen_element = element(Chosen_pos, Input), Last_element = element(Length, Input), Input_2 = setelement(Chosen_pos, Input, Last_element), Output_2 = setelement(Length, Output, Chosen_element), list_rearrange(F_2, Input, Length - 1, Output_2) . randomizer(1) -> {fun randomizer/1, 1}; randomizer(N) -> {fun randomizer/1, crypto:rand_uniform(1, N)}. But in the real world, the compiler does not realize, that we never _can_ touch the original tuples given to list_rearrange/4 after the setelement calls. list_rearrange/4 is not exported, tail recursive, and in the same module only used as the _last_ statement in the function that creates the tuples in its body. The compiler produces code to copy the tuples in each call making list_rearrange/4 O(length(List)^2)... - -- Allan Wegan Jabber: allanwegan@REDACTED ICQ: 209459114 Phone: +49 40 6732962 (Hansenet) Sch?neberger Strasse 60, 22149 Hamburg -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) iQEcBAEBAgAGBQJMAKPzAAoJENm5axHh7AcxrCwH/RKjyRJ1JacHZ4e5kZ1pPAMK /oFsaYTfjpBbQLsJhRCQuaHkSPDkA1k2YHqbNRiuXuPCPW5eazcE5kmUM2c2IJW8 EeSThk3jWEVmnGmTZVOEdjOGqmp74izuhWiRlBxs9TceT9Ejnzth5Dhe9zcsqGdi CDvj5A0oZHOv6EjKR/8FShGI1AxYQcIXgBvzKdomHeaXPPDzj3AKoDixSIWQgQoH 1joia4xXEmMz0ygsOhoMgXw2/plrAZWlr/WRLA7P97nG9Gp8WQ7dyPFcrWNu9KVY yx7ak/hrPJLAlx6j+foo7QoFC8CFL8ljUelmca0gw2/msmGb7aOwrT9OlcWlXzo= =qbql -----END PGP SIGNATURE----- From max.lapshin@REDACTED Sat May 29 11:43:50 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 29 May 2010 13:43:50 +0400 Subject: Good practice for binary distributing Message-ID: Hi! As erlyvideo is growing to commercial distributing of code, I've got new problem. I need to package some specific parts of functionality into packages without source code. Questions are: 1) is it possible to pack whole application into one java-like jar file, where all required beams and app file will be? 2) what are practices for compiling modules into such binary distribution: remove debug, etc. ? From jacob.vorreuter@REDACTED Sat May 29 21:55:01 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Sat, 29 May 2010 12:55:01 -0700 Subject: compiling linked-in driver on Mac OSX Message-ID: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> I'm having trouble compiling and loading the example driver from http://www.erlang.org/doc/tutorial/c_portdriver.html This is the driver source that I'm using: http://gist.github.com/418491 I'm using the gcc flags mentioned in other posts for compiling and linking on Mac OSX: $ gcc -o example_drv.so example_drv.c -bundle -flat_namespace -undefined suppress -I/usr/local/lib/erlang/erts-5.7.5/include/ Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> erl_ddll:load(".", "example_drv"). {error,{open_error,-10}} 2> erl_ddll:format_error({open_error,-10}). "dlopen(./example_drv.so, 2): no suitable image found. Did find:\n\t./example_drv.so: mach-o, but wrong architecture" Any ideas as to why I'm getting this "wrong architecture" error when trying to load the driver? I'm using R13B04 on Snow Leopard. Thanks, Jake From max.lapshin@REDACTED Sat May 29 22:05:00 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 30 May 2010 00:05:00 +0400 Subject: [erlang-questions] compiling linked-in driver on Mac OSX In-Reply-To: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> References: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> Message-ID: -arch x86_64 -fPIC -bundle -flat_namespace -undefined suppress From jacob.vorreuter@REDACTED Sat May 29 22:21:19 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Sat, 29 May 2010 13:21:19 -0700 Subject: [erlang-questions] compiling linked-in driver on Mac OSX In-Reply-To: References: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> Message-ID: Good suggestion, but it didn't seem to make a difference. I tried: gcc -o example_drv.so example_drv.c -arch x86_64 -fPIC -bundle -flat_namespace -undefined suppress -I/usr/local/lib/erlang/erts-5.7.5/include/ same result: Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> erl_ddll:load(".", "example_drv"). {error,{open_error,-10}} 2> erl_ddll:format_error({open_error,-10}). "dlopen(./example_drv.so, 2): no suitable image found. Did find:\n\t./example_drv.so: mach-o, but wrong architecture" On May 29, 2010, at 1:05 PM, Max Lapshin wrote: > -arch x86_64 -fPIC -bundle -flat_namespace -undefined suppress From max.lapshin@REDACTED Sat May 29 22:25:19 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 30 May 2010 00:25:19 +0400 Subject: [erlang-questions] compiling linked-in driver on Mac OSX In-Reply-To: References: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> Message-ID: maxbp:~ max$ file /usr/local/Cellar/erlang/R13B04/lib/erlang/erts-5.7.5/bin/erlexec /usr/local/Cellar/erlang/R13B04/lib/erlang/erts-5.7.5/bin/erlexec: Mach-O 64-bit executable x86_64 which architectures do you have? From jacob.vorreuter@REDACTED Sat May 29 22:30:44 2010 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Sat, 29 May 2010 13:30:44 -0700 Subject: [erlang-questions] compiling linked-in driver on Mac OSX In-Reply-To: References: <570DF159-61C3-4EF4-B79D-07A31FDAD06D@gmail.com> Message-ID: <14BC7AF9-BD87-4880-B39C-9627635DF0B9@gmail.com> Ah, i386 I guess. I tried that before but didn't format the error to see that my C code is just broken. Thanks Max! $ file erlexec erlexec: Mach-O executable i386 $ gcc -o example_drv.so example_drv.c -arch i386 -fPIC -bundle -flat_namespace -undefined suppress -I/usr/local/lib/erlang/erts-5.7.5/include/ Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> erl_ddll:load(".", "example_drv"). {error,{open_error,-10}} 2> erl_ddll:format_error({open_error,-10}). "dlopen(./example_drv.so, 2): Symbol not found: _bar\n Referenced from: /Users/jkvor/dev/basic_driver/c/example_drv.so\n Expected in: flat namespace\n in /Users/jkvor/dev/basic_driver/c/example_drv.so" On May 29, 2010, at 1:25 PM, Max Lapshin wrote: > maxbp:~ max$ file > /usr/local/Cellar/erlang/R13B04/lib/erlang/erts-5.7.5/bin/erlexec > /usr/local/Cellar/erlang/R13B04/lib/erlang/erts-5.7.5/bin/erlexec: > Mach-O 64-bit executable x86_64 > > > which architectures do you have? From hd2010@REDACTED Sun May 30 02:26:27 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 30 May 2010 02:26:27 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> Message-ID: <4C01B0B3.3000106@eonblast.com> Hi James, can you clarify this for me, what you mean by "shared" ... I think I understood the rest of your post alright, thanks! Henning > X = [57|List] > This creates a single cons (2 cells). The first element is 57. The second is > a pointer to List. So even though you've got two lists (X and List), all of > the data is shared except for the first cons of X. > > From hd2010@REDACTED Sun May 30 02:40:06 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 30 May 2010 02:40:06 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: <4C01B3E6.9000207@eonblast.com> Thanks Robert V, Robert R, James, Richard, thanks Dmitry: I did a naive benchmark on this meanwhile: http://www.eonblast.com/blog/cost-of-mutation/ From what I understood now, there is a clear and simple answer to my basic question: prefer deeper nesting over more flat structures. The benchmark compares structures like tupelMutateFlat_10(FlatTupel, NewD) -> { A, B, C, _, E, F, G, H, I, J } = FlatTupel, { A, B, C, NewD, E, F, G, H, I, J }. tupelMutateNested_10(NestedTupel, NewD) -> { { AB, {C, _}}, EFGH, IJ } = NestedTupel, { { AB, {C, NewD}}, EFGH, IJ }. Any comment on erroneous benchmark construction highly welcome. Specifically in answer to Robert Virding: > If you modify one of the nodes then > the only other nodes you will have to rewrite are those nodes between > the root of the tree and the modified node. Plus the sibling nodes, I think, which were my focus. > What sized nodes depends entirely upon what you are doing, the only > general hint I can give is use that which fits best into your > application. I mean it, really. Don't go complicating matters by > picking weirdo data structures. > I was asking this general question to have guidance on the basic decisions in application and data design. > The only time I would seriously start thinking about what size thing > to use is if you are writing more general library type packages. > Yes. Plus, I think, what structure the data should have that you write away to disk would equally affect an app performance so profoundly that it may warrant this kind of reflection. > Generally having smaller nodes but a deeper resulting tree will lead > to less rewriting, but as the tree is deeper it will cost more to > traverse it and though the rewrites will be smaller there will be more > of them. How strong that effect is, that is what I test in the above benchmark. In the extreme, it gets pretty high: 100 element flat list const Bin (discarding result, 1M ): run 1450 wall 1512 ms. 100 element flat list integer N (discarding result, 1M ): run 1470 wall 1550 ms. 100 element nested list const Bin (discarding result, 1M ): run 400 wall 462 ms. (72% faster) 100 element nested list integer N (discarding result, 1M ): run 380 wall 414 ms. (74% faster) Let me know please if you find where the assumptions are wrong. On the traversal statement, Rob, I can't immediately follow --- maybe you mean with a lot larger structures? I am talking pretty much about a usual data base record size, say, of a customer with a lot of associated data attached. Thanks, Henning From hd2010@REDACTED Sun May 30 02:44:52 2010 From: hd2010@REDACTED (Henning Diedrich) Date: Sun, 30 May 2010 02:44:52 +0200 Subject: [erlang-questions] Cost of Copy In-Reply-To: References: <4BFE85F9.3010705@eonblast.com> <4BFED171.1020100@eonblast.com> Message-ID: <4C01B504.1040108@eonblast.com> Hi Johan, Johan Montelius wrote: > If we look at larger structures the difference become apparent. As per the mailing list, I did a benchmark on this meanwhile. > > If we instead represent this as a list of length N we would have to > construct a new list to the point where we have located the element > that we want to change. > In average thins means N/2 new list cells for each update. This is for > large N more efficient than having to construct a new tuple of size N. This is something I didn't dig deeper yet, thanks a lot for clarifying the difference that lists make. Thanks a lot for your explanations, Henning From sysop@REDACTED Sun May 30 04:57:10 2010 From: sysop@REDACTED (Matt Stancliff) Date: Sat, 29 May 2010 19:57:10 -0700 Subject: [erlang-questions] Good practice for binary distributing In-Reply-To: References: Message-ID: <60482D9E-777C-4A03-9C8B-C22DA386CB55@mindspring.com> Hey Max, On May 29, 2010, at 2:43 AM, Max Lapshin wrote: > Questions are: > 1) is it possible to pack whole application into one java-like jar > file, where all required beams and app file will be? Check out "Loading of Code From Archive Files" at http://www.erlang.org/doc/man/code.html Though, everybody should be using rebar to build and distribute erlang apps: http://hg.basho.com/rebar/src Use the rebar control scripts. Use reltool with the help of rebar. Use the rebar/OTP directory layout. Use the rebar testing hooks. Use the rebar management of your dependencies (no git submodules). Rebar, oddly, has no docs though. Check out some other projects using it for how to layout your code. Examples: http://hg.basho.com/riak/src http://hg.basho.com/innostore/src I rebar-ized OMS last month. Take a look to see how things were laid out before and after. Before (autotools): http://github.com/mattsta/oms-matt/tree/5b50c1 After (rebar): http://github.com/mattsta/oms-matt/ Same with an erlang redis library: Before (Emakefile, Makefile, Rakefile, rebar): http://github.com/mattsta/er/tree/20bcb6 After (rebar-only): http://github.com/mattsta/er/ > 2) what are practices for compiling modules into such binary > distribution: remove debug, etc. ? Look at http://www.erlang.org/doc/man/beam_lib.html You can encrypt your debug info so it's still usable by you, but the source can't be reconstructed by others. -Matt -- Matt Stancliff San Jose, CA @mattsta iPhone: 678-591-9337 "The best way to predict the future is to invent it." --Alan Kay From max.lapshin@REDACTED Sun May 30 08:50:44 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 30 May 2010 10:50:44 +0400 Subject: [erlang-questions] Good practice for binary distributing In-Reply-To: <60482D9E-777C-4A03-9C8B-C22DA386CB55@mindspring.com> References: <60482D9E-777C-4A03-9C8B-C22DA386CB55@mindspring.com> Message-ID: On Sun, May 30, 2010 at 6:57 AM, Matt Stancliff wrote: > Same with an erlang redis library: > ?Before (Emakefile, Makefile, Rakefile, rebar): > http://github.com/mattsta/er/tree/20bcb6 > ?After (rebar-only): http://github.com/mattsta/er/ > It is nice, but I don't understand, how will it help me to build debian packages. From kenji.rikitake@REDACTED Sun May 30 09:53:20 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Sun, 30 May 2010 16:53:20 +0900 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> Message-ID: <20100530075320.GA29981@k2r.org> In the message <5DD3CDF1-6148-46FC-8496-A3010B97BD30@REDACTED> dated Fri, May 28, 2010 at 10:56:45AM +1200, Richard O'Keefe writes: [...] > It might be a good idea to move the core random number generator > in random: into C (at present random:uniform() is quite slow) and > perhaps even replace it by a better RNG. [...] For a testing purpose, the RNG doesn't really have to be cryptographycally safe; it only has to have a uniform distribution. Anybody has implemented a NIF or port version of Mersenne Twister or SFMT? http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html (Mersenne Twister has been incorporated as the default RNG for Python and R, so far as I know.) BTW for a native Erlang version of MT: http://www.erlang.org/pipermail/erlang-questions/2007-June/027065.html Regards, Kenji Rikitake From g@REDACTED Sun May 30 15:26:36 2010 From: g@REDACTED (Garrett Smith) Date: Sun, 30 May 2010 08:26:36 -0500 Subject: [erlang-questions] Good practice for binary distributing In-Reply-To: References: <60482D9E-777C-4A03-9C8B-C22DA386CB55@mindspring.com> Message-ID: On Sun, May 30, 2010 at 1:50 AM, Max Lapshin wrote: > On Sun, May 30, 2010 at 6:57 AM, Matt Stancliff wrote: >> Same with an erlang redis library: >> ?Before (Emakefile, Makefile, Rakefile, rebar): >> http://github.com/mattsta/er/tree/20bcb6 >> ?After (rebar-only): http://github.com/mattsta/er/ >> > > It is nice, but I don't understand, how will it help me to build > debian packages. I suspect you probably have some work ahead of you as the build tools in Erlang world are still relatively young. Take a look at the rabbitmq project. They provide a full assortment of system packages and you might be able to borrow some ideas. Garrett From james.hague@REDACTED Sun May 30 19:55:53 2010 From: james.hague@REDACTED (James Hague) Date: Sun, 30 May 2010 12:55:53 -0500 Subject: [erlang-questions] Cost of Copy In-Reply-To: <4C01B0B3.3000106@eonblast.com> References: <4BFE85F9.3010705@eonblast.com> <4C01B0B3.3000106@eonblast.com> Message-ID: A cons (to use the Lisp terminology) is a "something" and a pointer. In the [57|List] example, one cons is created where the something is 57 and the pointer is List. That's it. Two memory cells. On Sat, May 29, 2010 at 7:26 PM, Henning Diedrich wrote: > Hi James, > > can you clarify this for me, what you mean by "shared" ... I think I > understood the rest of your post alright, thanks! > > Henning > > > X = [57|List] > This creates a single cons (2 cells). The first element is 57. The second is > a pointer to List. So even though you've got two lists (X and List), all of > the data is shared except for the first cons of X. > > > > > From ok@REDACTED Mon May 31 03:08:53 2010 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 31 May 2010 13:08:53 +1200 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: <20100530075320.GA29981@k2r.org> References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org> Message-ID: On May 30, 2010, at 7:53 PM, Kenji Rikitake wrote: > In the message <5DD3CDF1-6148-46FC-8496-A3010B97BD30@REDACTED> > dated Fri, May 28, 2010 at 10:56:45AM +1200, > Richard O'Keefe writes: > [...] >> It might be a good idea to move the core random number generator >> in random: into C (at present random:uniform() is quite slow) and >> perhaps even replace it by a better RNG. > [...] > > For a testing purpose, the RNG doesn't really have to be > cryptographycally safe; it only has to have a uniform distribution. Where did "cryptographically safe" come from? By better, I meant - faster - having better statistical properties > > Anybody has implemented a NIF or port version of Mersenne Twister or > SFMT? > http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html and those are excellent examples of what I meant. From silent_vendetta@REDACTED Mon May 31 04:55:22 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Sun, 30 May 2010 19:55:22 -0700 Subject: Error inserting record into Mnesia In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org>, Message-ID: Can anyone point out to me what might be going wrong with the following code? I can't figure out why I keep getting the bad_type error. -record(active_entity, {id=undefined, components=[], debug_components=[]}). 19> Act = #active_entity{}.#active_entity{id = undefined,components = [], debug_components = []}20> mnesia:create_table(active_entity, [{ram_copies, [node()]}, {type, ordered_set}]). {atomic,ok}21> mnesia:dirty_write(active_entity, Act). ** exception exit: {aborted, {bad_type, #active_entity{ id = undefined,components = [], debug_components = []}}} in function mnesia:abort/1 _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 From igorrs@REDACTED Mon May 31 05:51:05 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 31 May 2010 00:51:05 -0300 Subject: [erlang-questions] Error inserting record into Mnesia In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org> Message-ID: I think it's because you didn't specify the {attributes, [id, components, debug_components]} argument when creating the table. On Sun, May 30, 2010 at 11:55 PM, Chris Hicks wrote: > > Can anyone point out to me what might be going wrong with the following code? I can't figure out why I keep getting the bad_type error. > -record(active_entity, {id=undefined, components=[], debug_components=[]}). > 19> Act = #active_entity{}.#active_entity{id = undefined,components = [], ? ? ? ? ? ? ? debug_components = []}20> mnesia:create_table(active_entity, [{ram_copies, [node()]}, {type, ordered_set}]). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{atomic,ok}21> mnesia:dirty_write(active_entity, Act). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?** exception exit: {aborted, ? ? ? ? ? ? ? ? ? ? ? {bad_type, ? ? ? ? ? ? ? ? ? ? ? ? ? #active_entity{ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? id = undefined,components = [], ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? debug_components = []}}} ? ? in function ?mnesia:abort/1 > _________________________________________________________________ > The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. > http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From silent_vendetta@REDACTED Mon May 31 05:59:32 2010 From: silent_vendetta@REDACTED (Chris Hicks) Date: Sun, 30 May 2010 20:59:32 -0700 Subject: [erlang-questions] Error inserting record into Mnesia In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> ,<6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au>,<5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org>, , Message-ID: Gah, I should have known that. Feel stupid for having hacked away for half an hour and scanning the docs just to have missed that one piece. Thank you. > From: igorrs@REDACTED > Date: Mon, 31 May 2010 00:51:05 -0300 > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Error inserting record into Mnesia > > I think it's because you didn't specify the {attributes, [id, > components, debug_components]} argument when creating the table. > > On Sun, May 30, 2010 at 11:55 PM, Chris Hicks > wrote: > > > > Can anyone point out to me what might be going wrong with the following code? I can't figure out why I keep getting the bad_type error. > > -record(active_entity, {id=undefined, components=[], debug_components=[]}). > > 19> Act = #active_entity{}.#active_entity{id = undefined,components = [], debug_components = []}20> mnesia:create_table(active_entity, [{ram_copies, [node()]}, {type, ordered_set}]). {atomic,ok}21> mnesia:dirty_write(active_entity, Act). ** exception exit: {aborted, {bad_type, #active_entity{ id = undefined,components = [], debug_components = []}}} in function mnesia:abort/1 > > _________________________________________________________________ > > The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. > > http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > _________________________________________________________________ The New Busy is not the old busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3 From rpettit@REDACTED Mon May 31 07:23:45 2010 From: rpettit@REDACTED (Rick Pettit) Date: Mon, 31 May 2010 00:23:45 -0500 Subject: [erlang-questions] Error inserting record into Mnesia In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org> Message-ID: <14a860a42905aafd7a0d929a960a1f5c.squirrel@squirrelmail.vail> On Sun, May 30, 2010 10:51 pm, Igor Ribeiro Sucupira wrote: > I think it's because you didn't specify the {attributes, [id, > components, debug_components]} argument when creating the table. Since he has a record defined I would suggest the following instead: {attributes, record_info(fields, active_entity)} That way you don't have to remember to update both record and table definitions should the record change in the future. -Rick > On Sun, May 30, 2010 at 11:55 PM, Chris Hicks > wrote: >> >> Can anyone point out to me what might be going wrong with the following >> code? I can't figure out why I keep getting the bad_type error. >> -record(active_entity, {id=undefined, components=[], >> debug_components=[]}). >> 19> Act = #active_entity{}.#active_entity{id = undefined,components = >> [], ? ? ? ? ? ? ? debug_components = []}20> >> mnesia:create_table(active_entity, [{ram_copies, [node()]}, {type, >> ordered_set}]). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{atomic,ok}21> >> mnesia:dirty_write(active_entity, Act). ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? >> ? ? ?** exception exit: {aborted, ? ? ? ? ? ? ? ? ? ? ? {bad_type, ? ? ? >> ? ? ? ? ? ? ? ? ? ? #active_entity{ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? id = >> undefined,components = [], ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? >> debug_components = []}}} ? ? in function ?mnesia:abort/1 >> _________________________________________________________________ >> The New Busy is not the too busy. Combine all your e-mail accounts with >> Hotmail. >> http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From raimo+erlang-questions@REDACTED Mon May 31 09:29:57 2010 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 31 May 2010 09:29:57 +0200 Subject: [erlang-questions] Freezing a process In-Reply-To: <4C008F4A.1080800@erlang-solutions.com> References: <4C008F4A.1080800@erlang-solutions.com> Message-ID: <20100531072957.GA6170@erix.ericsson.se> On Sat, May 29, 2010 at 05:51:38AM +0200, Ulf Wiger wrote: > On 05/29/2010 12:18 AM, Jeffrey Rennie (?????????) wrote: > > > > In other light-weight concurrency solutions I've seen, such as > > stackless Python, it's straight-forward to serialize processes. I was > > afraid I had overlooked something in Erlang. > > In Termite (http://code.google.com/p/termite/), which is a Scheme > dialect inspired by Erlang, it is also possible to migrate a > process. > > In Erlang, it isn't possible to do it in a generic way, and > transparently. There are a few details that get in the way, such > as that the owner of a port must be a local process, and ports > are by design not possible to move transparently. > > This is actually more or less an original design philosophy of > Erlang. It was designed to support distribution transparency, > such that it shouldn't matter where the other processes are, > that are talking to you. At the same time, it had to be > possible to be very explicit about where you spawn a process, > since, if a process controls a hardware device, it should be > running in a place where this is at all possible to do. > > Another issue is that new failure conditions arise when > crossing a node boundary (such as disrupted communication and > partial failure), and it is sometimes necessary to take this > into account in your program. > > These things are not necessarily anathema to allowing the > migration of processes. One could imagine a process flag > forbidding the migration, for cases where the programmer > thinks that code would break. But the OTP design philosophy > is rather to move whole applications and being rather > explicit about node boundaries, such that an OTP application > instance always belongs to a specific node. > > OTOH, if a process is designed to be easily movable, in that > it has no tricky local dependencies, it is trivial to write > a migrate() function for it. If it is a behaviour, the sys > module provides the means to suspend a process, extract its > state, and resume it. But in the case of migration, you would > rather start the process in its new location with the > migrated state, and e.g. use the gen_server:enter_loop() > function to resume it. Beware of the complication of links and monitors since there is no mechanism to move a link nor a monitor. A link could be moved by unlinking from the moving process and then linking from the new moved process, but if the other link end process has state that expects a link to a certain pid() it will be surprised. A monitor can not be moved since it is controlled from the monitoring process. When your moving process dies (I assume it will die after being moved), the monitor will be triggered, and the monitoring process will have to know there is a new process to monitor. If, e.g, the moving process is a gen_Something, all clients calling it will set up temporary monitors during the request and that client code will be surprised if the gen_Something disappears during a call... > > BR, > Ulf W > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dmitriid@REDACTED Mon May 31 10:51:14 2010 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Mon, 31 May 2010 11:51:14 +0300 Subject: Erlang-related banners/badges for your site Message-ID: Hi, all If you ever wanted an Erlang-related banner//badge for your site that would read something like "Erlang FTW!!!", you can now have one: http://banners.erlanger.ru/ Three sizes, two themes, customized labels. I'm open to suggestions (especially with regard to fonts, I have a problem rendering them correctly on Ubuntu 8.04 Server with Python's PIL) P.S. Since many confuse erlanger with enlarger, Russian Erlang community now has a sort of meme: "erlang your code" :) From zabrane3@REDACTED Mon May 31 12:20:16 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 31 May 2010 12:20:16 +0200 Subject: Erlang "system_limit" trouble! Message-ID: Hi guys, One of my modules ("report.erl") generates a lot (~100K) of daily reports on disk. I'm facing an issue with the following code: create_disk_report(Path, Bin) -> ok = file:write_file(Path, Bin). ... {"init terminating in do_boot",{{badmatch,{error,system_limit}},[{report,create_disk_report,2} ...}} "file:write_file" doesn't seem to return "ok" in my case (it returns {error, system_limit)}. I even tried: ok = file:write_file(Path, Bin, [write, raw, binary]). But it fails too, and the same error persist. What am i doing wrong guys? My config -------------- Exactly the same error appears on both OSX and Linux. 1) OSX 10.5.8 * Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] * gcc version 4.0.1 (Apple Inc. build 5488) * OS ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 4096 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 266 virtual memory (kbytes, -v) unlimited 2) Ubuntu 9.04 (jaunty) * Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] * gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3 * ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 20 file size (blocks, -f) unlimited pending signals (-i) 16382 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited 3) OpenSuSE * Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] * gcc (SUSE Linux) 4.4.1 [gcc-4_4-branch revision 150839] * ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 24242 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) 2642732 open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 24242 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited -- Regards Zabrane From francesco@REDACTED Mon May 31 12:36:16 2010 From: francesco@REDACTED (Francesco Cesarini (Erlang Solutions)) Date: Mon, 31 May 2010 11:36:16 +0100 Subject: June 2010 London Erlang Factory, Last Day for Early Bird Registration Message-ID: <4C039120.3000709@erlang-solutions.com> Hi All, A reminder that today is the last day for the Early Bird Registration for the 2010 Erlang Factory. Dates for the conference are the 10th and 11th of June, while the training takes place the 7th - 9th. Register by midnight today to save 200GBP on the conference fees and the training courses. Read more about it on the Factory site: http://erlang-factory.com/conference/London2010 http://erlang-factory.com/conference/London2010/tracks http://erlang-factory.com/conference/London2010/university Highlights from this year is a joint keynote by Robert Virding and Joe Armstrong. Martin Odersky will talk about the influences of Scala on Erlang, alongside talks on Zotonic, Erjang, Efene, RabbitMQ, Riak, CouchDB, Wrangler and QuickCheck, to mention but a few. With over 50 speakers, six tracks and four courses preceeding the conference, this event will undoubtledly be the largest Erlang event of the year which should not be missed. Hope to see you there, Francesco --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From gleber.p@REDACTED Mon May 31 12:46:02 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 31 May 2010 12:46:02 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: This should be helpful: http://www.erlang.org/doc/efficiency_guide/advanced.html#ports Do you keep these daily reports files open? Could it be that these files are not closed properly? On Mon, May 31, 2010 at 12:20, zabrane Mikael wrote: > Hi guys, > > One of my modules ("report.erl") generates a lot ?(~100K) of daily reports > on disk. > I'm facing an issue with the following code: > > create_disk_report(Path, Bin) -> > ?ok = file:write_file(Path, Bin). > > ... > {"init terminating in > do_boot",{{badmatch,{error,system_limit}},[{report,create_disk_report,2} > ...}} > > "file:write_file" doesn't seem to return "ok" in my case (it returns {error, > system_limit)}. > > I even tried: > ok = file:write_file(Path, Bin, [write, raw, binary]). > > But it fails too, and the same error persist. > What am i doing wrong guys? > > > My config > -------------- > Exactly the same error appears on both OSX and Linux. > > 1) OSX 10.5.8 > * Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] > [kernel-poll:false] > * gcc version 4.0.1 (Apple Inc. build 5488) > * OS ulimit -a > core file size ? ? ? ? ?(blocks, -c) 0 > data seg size ? ? ? ? ? (kbytes, -d) 6144 > file size ? ? ? ? ? ? ? (blocks, -f) unlimited > max locked memory ? ? ? (kbytes, -l) unlimited > max memory size ? ? ? ? (kbytes, -m) unlimited > open files ? ? ? ? ? ? ? ? ? ? ?(-n) 4096 > pipe size ? ? ? ? ? ?(512 bytes, -p) 1 > stack size ? ? ? ? ? ? ?(kbytes, -s) 8192 > cpu time ? ? ? ? ? ? ? (seconds, -t) unlimited > max user processes ? ? ? ? ? ? ?(-u) 266 > virtual memory ? ? ? ? ?(kbytes, -v) unlimited > > 2) Ubuntu 9.04 (jaunty) > * Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > * gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3 > * ulimit -a > core file size ? ? ? ? ?(blocks, -c) 0 > data seg size ? ? ? ? ? (kbytes, -d) unlimited > scheduling priority ? ? ? ? ? ? (-e) 20 > file size ? ? ? ? ? ? ? (blocks, -f) unlimited > pending signals ? ? ? ? ? ? ? ? (-i) 16382 > max locked memory ? ? ? (kbytes, -l) 64 > max memory size ? ? ? ? (kbytes, -m) unlimited > open files ? ? ? ? ? ? ? ? ? ? ?(-n) 1024 > pipe size ? ? ? ? ? ?(512 bytes, -p) 8 > POSIX message queues ? ? (bytes, -q) 819200 > real-time priority ? ? ? ? ? ? ?(-r) 0 > stack size ? ? ? ? ? ? ?(kbytes, -s) 8192 > cpu time ? ? ? ? ? ? ? (seconds, -t) unlimited > max user processes ? ? ? ? ? ? ?(-u) unlimited > virtual memory ? ? ? ? ?(kbytes, -v) unlimited > file locks ? ? ? ? ? ? ? ? ? ? ?(-x) unlimited > > 3) OpenSuSE > * Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > * gcc (SUSE Linux) 4.4.1 [gcc-4_4-branch revision 150839] > * ulimit -a > core file size ? ? ? ? ?(blocks, -c) 0 > data seg size ? ? ? ? ? (kbytes, -d) unlimited > scheduling priority ? ? ? ? ? ? (-e) 0 > file size ? ? ? ? ? ? ? (blocks, -f) unlimited > pending signals ? ? ? ? ? ? ? ? (-i) 24242 > max locked memory ? ? ? (kbytes, -l) 64 > max memory size ? ? ? ? (kbytes, -m) 2642732 > open files ? ? ? ? ? ? ? ? ? ? ?(-n) 1024 > pipe size ? ? ? ? ? ?(512 bytes, -p) 8 > POSIX message queues ? ? (bytes, -q) 819200 > real-time priority ? ? ? ? ? ? ?(-r) 0 > stack size ? ? ? ? ? ? ?(kbytes, -s) 8192 > cpu time ? ? ? ? ? ? ? (seconds, -t) unlimited > max user processes ? ? ? ? ? ? ?(-u) 24242 > virtual memory ? ? ? ? ?(kbytes, -v) unlimited > file locks ? ? ? ? ? ? ? ? ? ? ?(-x) unlimited > > -- > Regards > Zabrane > From zabrane3@REDACTED Mon May 31 13:00:18 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 31 May 2010 13:00:18 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: 2010/5/31 Gleb Peregud > This should be helpful: > > http://www.erlang.org/doc/efficiency_guide/advanced.html#ports Already looked to that, but don't really now what to try ... thanks Gleb. > Do you keep these daily reports files open? No, they remain closed. When everything is done, a bash script copy them to another machine late by night. > Could it be that these files are not closed properly? Here, I'm assuming that "file:write_file" close the files properly. Is this true? From ivan@REDACTED Mon May 31 13:03:07 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 31 May 2010 12:03:07 +0100 Subject: Reading and Writing .wav files Message-ID: <4C03976B.4050501@llaisdy.com> Dear All Please find below wave.erl, containing two functions to read and write .wav files. wave:read(FileName) reads a wav file and returns a record containing the audio sample rate and the raw data; wave:write(WavRecord, FileName) does the opposite. The script was an experiment in parsing and packing binaries and I'm very pleased at how straightforward it was (mostly: I did ask for help here for one point) and how clearly readable the code is. It all works (tested on mono and stereo wav files). There are a couple of ancillary functions, which perhaps re-implement functions already in the standard library, or are otherwise sub-optimal: lp2pl converts a list of pairs into a pair of lists, and lt2l does (almost) the opposite, converting a list of tuples into a flat list. Comments welcome. Best wishes Ivan % ----- wave.erl -module(wave). -export([read/1, write/2, write/3]). %% Erlang package to read and write .wav files. %% wave record format %% version 1.0: %% - does not support compression %% - only pcm encoding (AudioFormat = 1) % wav file data always little-endian % data is list of lists of integers (one list for each channel) -record(wave, {audio_format, sample_rate, data}). read(FileName) -> %% returns wave record {ok, Binary} = file:read_file(FileName), <> = Binary, % TODO: test file for well-formedness, e.g.: % ChunkID == "RIFF" % ChunkSize == SubChunk2Size + 36 % Format == "WAVE" % AudioFormat = 1 % no other encodings supported in v1.0 % SubChunk1ID == "fmt " % SubChunk1Size == 16 % SubChunk2ID == "data" % SubChunk2Size == NumSamples * NumChannels * BitsPerSample div 8 Channels = data2channels(Data, NumChannels, BitsPerSample), io:format("* ~p~n~n- ChunkID: ~p~n- ChunkSize: ~B~n- Format: ~p~n- SubChunk1ID: ~p~n- SubChunk1Size: ~B~n- AudioFormat: ~B~n- NumChannels: ~B~n- SampleRate: ~B~n- ByteRate: ~B~n- BlockAlign: ~B~n- BitsPerSample: ~B~n- SubChunk2ID: ~p~n- SubChunk2Size: ~B~n- Data: [...]~n~n", [FileName, binary_to_list(ChunkID), ChunkSize, binary_to_list(Format), binary_to_list(SubChunk1ID), SubChunk1Size, AudioFormat, NumChannels, SampleRate, ByteRate, BlockAlign, BitsPerSample, binary_to_list(SubChunk2ID), SubChunk2Size ]), #wave{sample_rate=SampleRate, audio_format=AudioFormat, data=Channels }. mono(Data, NBytes) -> [[ X || <> <= Data ]]. stereo(Data, NBytes) -> LP = [ [A,B] || << A:NBytes/little-signed-integer-unit:8, B:NBytes/little-signed-integer-unit:8 >> <= Data], lp2pl(LP, [], []). % list of pairs to pair of lists lp2pl([], Left, Right) -> [lists:reverse(Left), lists:reverse(Right)]; lp2pl([[L,R]|T], Left, Right) -> lp2pl(T, [L|Left], [R|Right]). data2channels(Data, NumChannels, BitsPerSample) -> NBytes = BitsPerSample div 8, case NumChannels of 1 -> mono(Data, NBytes); 2 -> stereo(Data, NBytes) end. lt2l([], Acc) -> lists:reverse(Acc); lt2l([H|T], Acc) -> lt2l(T, [element(2, H), element(1, H) | Acc]). pl2l(PL) -> X = lists:zip(lists:nth(1, PL), lists:nth(2, PL)), lt2l(X, []). channels2dataBin(Channels, BitsPerSample) -> case length(Channels) of 1 -> Data = lists:nth(1, Channels); 2 -> Data = pl2l(Channels) end, Size = length(Data) * (BitsPerSample div 8), DataBin = list_to_binary([ <> || X <- Data]), {Size, DataBin}. write(WavRecord, FileName) -> write(WavRecord, FileName, 16). write(WavRecord, FileName, BitsPerSample) -> {SubChunk2Size, Data} = channels2dataBin(WavRecord#wave.data, BitsPerSample), ChunkID = list_to_binary("RIFF"), Format = list_to_binary("WAVE"), SubChunk1ID = list_to_binary("fmt "), SubChunk1Size = 16, AudioFormat = 1, % only pcm encoding supported NumChannels = length(WavRecord#wave.data), SampleRate = WavRecord#wave.sample_rate, ByteRate = SampleRate * NumChannels * BitsPerSample div 8, BlockAlign = NumChannels * BitsPerSample div 8, SubChunk2ID = list_to_binary("data"), ChunkSize = SubChunk2Size + 36, io:format("* ~p~n~n- ChunkID: ~p~n- ChunkSize: ~B~n- Format: ~p~n- SubChunk1ID: ~p~n- SubChunk1Size: ~B~n- AudioFormat: ~B~n- NumChannels: ~B~n- SampleRate: ~B~n- ByteRate: ~B~n- BlockAlign: ~B~n- BitsPerSample: ~B~n- SubChunk2ID: ~p~n- SubChunk2Size: ~B~n- Data: [...]~n~n", [FileName, binary_to_list(ChunkID), ChunkSize, binary_to_list(Format), binary_to_list(SubChunk1ID), SubChunk1Size, AudioFormat, NumChannels, SampleRate, ByteRate, BlockAlign, BitsPerSample, binary_to_list(SubChunk2ID), SubChunk2Size ]), Binary = <>, file:write_file(FileName, Binary), {ok, FileName}. % ----- wave.erl -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From gleber.p@REDACTED Mon May 31 13:09:39 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 31 May 2010 13:09:39 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: Try using "lsof" with the process ID of the beam to see if these files are really closed properly On Mon, May 31, 2010 at 13:00, zabrane Mikael wrote: > 2010/5/31 Gleb Peregud >> >> This should be helpful: >> >> http://www.erlang.org/doc/efficiency_guide/advanced.html#ports > > Already looked to that, but don't really now what to try ... thanks Gleb. > >> >> Do you keep these daily reports files open? > > No, they remain closed. When everything is done, a bash script copy > them to another machine late by night. > >> >> Could it be that these?files are not closed properly? > > > Here, I'm assuming that "file:write_file" close the files properly. > Is this true? > From max.lapshin@REDACTED Mon May 31 13:13:59 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 31 May 2010 15:13:59 +0400 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C03976B.4050501@llaisdy.com> References: <4C03976B.4050501@llaisdy.com> Message-ID: So, if file is 400 MB size, it will return 400MB binary? Much more convenient is such protocol: wave:pread(File, Offset) -> {ok, header, Header} | {ok, frame, Frame} | eof Maybe so? From mazen.harake@REDACTED Mon May 31 13:29:21 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 31 May 2010 14:29:21 +0300 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C03976B.4050501@llaisdy.com> References: <4C03976B.4050501@llaisdy.com> Message-ID: <4C039D91.6050402@erlang-solutions.com> Perhaps you should always put a license on it... /Mazen On 31/05/2010 14:03, Ivan Uemlianin wrote: > Dear All > > Please find below wave.erl, containing two functions to read and write > .wav files. > > wave:read(FileName) reads a wav file and returns a record containing > the audio sample rate and the raw data; wave:write(WavRecord, > FileName) does the opposite. > > The script was an experiment in parsing and packing binaries and I'm > very pleased at how straightforward it was (mostly: I did ask for help > here for one point) and how clearly readable the code is. > > It all works (tested on mono and stereo wav files). > > There are a couple of ancillary functions, which perhaps re-implement > functions already in the standard library, or are otherwise > sub-optimal: lp2pl converts a list of pairs into a pair of lists, and > lt2l does (almost) the opposite, converting a list of tuples into a > flat list. > Comments welcome. > > Best wishes > > Ivan > > % ----- wave.erl > > -module(wave). > -export([read/1, write/2, write/3]). > > %% Erlang package to read and write .wav files. > > %% wave record format > %% version 1.0: > %% - does not support compression > %% - only pcm encoding (AudioFormat = 1) > > % wav file data always little-endian > % data is list of lists of integers (one list for each channel) > -record(wave, {audio_format, sample_rate, data}). > > read(FileName) -> > %% returns wave record > {ok, Binary} = file:read_file(FileName), > > < ChunkSize:4/little-unsigned-integer-unit:8, > Format:4/binary, > SubChunk1ID:4/binary, > SubChunk1Size:4/little-unsigned-integer-unit:8, > AudioFormat:2/little-unsigned-integer-unit:8, % 1 = pcm > NumChannels:2/little-unsigned-integer-unit:8, % 1 = mono > SampleRate:4/little-unsigned-integer-unit:8, % 16000, etc > ByteRate:4/little-unsigned-integer-unit:8, % SampleRate * > NumChannels * BitsPerSample/8 > BlockAlign:2/little-unsigned-integer-unit:8, % NumChannels * > BitsPerSample/8 > BitsPerSample:2/little-unsigned-integer-unit:8, % 16, etc > SubChunk2ID:4/binary, > SubChunk2Size:4/little-unsigned-integer-unit:8, > Data/binary>> = Binary, > > % TODO: test file for well-formedness, e.g.: > % ChunkID == "RIFF" > % ChunkSize == SubChunk2Size + 36 > % Format == "WAVE" > % AudioFormat = 1 % no other encodings supported in v1.0 > % SubChunk1ID == "fmt " > % SubChunk1Size == 16 > % SubChunk2ID == "data" > % SubChunk2Size == NumSamples * NumChannels * BitsPerSample div 8 > > Channels = data2channels(Data, NumChannels, BitsPerSample), > > io:format("* ~p~n~n- ChunkID: ~p~n- ChunkSize: ~B~n- Format: ~p~n- > SubChunk1ID: ~p~n- SubChunk1Size: ~B~n- AudioFormat: ~B~n- > NumChannels: ~B~n- SampleRate: ~B~n- ByteRate: ~B~n- BlockAlign: ~B~n- > BitsPerSample: ~B~n- SubChunk2ID: ~p~n- SubChunk2Size: ~B~n- Data: > [...]~n~n", > [FileName, > binary_to_list(ChunkID), > ChunkSize, > binary_to_list(Format), > binary_to_list(SubChunk1ID), > SubChunk1Size, > AudioFormat, > NumChannels, > SampleRate, > ByteRate, > BlockAlign, > BitsPerSample, > binary_to_list(SubChunk2ID), > SubChunk2Size > ]), > > #wave{sample_rate=SampleRate, > audio_format=AudioFormat, > data=Channels > }. > > mono(Data, NBytes) -> > [[ X || <> <= Data ]]. > > stereo(Data, NBytes) -> > LP = [ [A,B] || << A:NBytes/little-signed-integer-unit:8, > B:NBytes/little-signed-integer-unit:8 >> <= Data], > lp2pl(LP, [], []). > % list of pairs to pair of lists > lp2pl([], Left, Right) -> > [lists:reverse(Left), lists:reverse(Right)]; > > lp2pl([[L,R]|T], Left, Right) -> > lp2pl(T, [L|Left], [R|Right]). > > data2channels(Data, NumChannels, BitsPerSample) -> > NBytes = BitsPerSample div 8, > case NumChannels of > 1 -> mono(Data, NBytes); > 2 -> stereo(Data, NBytes) > end. > > lt2l([], Acc) -> > lists:reverse(Acc); > > lt2l([H|T], Acc) -> > lt2l(T, [element(2, H), element(1, H) | Acc]). > > pl2l(PL) -> > X = lists:zip(lists:nth(1, PL), lists:nth(2, PL)), > lt2l(X, []). > > channels2dataBin(Channels, BitsPerSample) -> > case length(Channels) of > 1 -> Data = lists:nth(1, Channels); > 2 -> Data = pl2l(Channels) > end, > Size = length(Data) * (BitsPerSample div 8), > DataBin = list_to_binary([ <> || > X <- Data]), > {Size, DataBin}. > > write(WavRecord, FileName) -> > write(WavRecord, FileName, 16). > > write(WavRecord, FileName, BitsPerSample) -> > {SubChunk2Size, Data} = channels2dataBin(WavRecord#wave.data, > BitsPerSample), > > ChunkID = list_to_binary("RIFF"), > Format = list_to_binary("WAVE"), > SubChunk1ID = list_to_binary("fmt "), > SubChunk1Size = 16, > AudioFormat = 1, % only pcm encoding supported > NumChannels = length(WavRecord#wave.data), > SampleRate = WavRecord#wave.sample_rate, > ByteRate = SampleRate * NumChannels * BitsPerSample div 8, > BlockAlign = NumChannels * BitsPerSample div 8, > SubChunk2ID = list_to_binary("data"), > ChunkSize = SubChunk2Size + 36, > > io:format("* ~p~n~n- ChunkID: ~p~n- ChunkSize: ~B~n- Format: ~p~n- > SubChunk1ID: ~p~n- SubChunk1Size: ~B~n- AudioFormat: ~B~n- > NumChannels: ~B~n- SampleRate: ~B~n- ByteRate: ~B~n- BlockAlign: ~B~n- > BitsPerSample: ~B~n- SubChunk2ID: ~p~n- SubChunk2Size: ~B~n- Data: > [...]~n~n", > [FileName, > binary_to_list(ChunkID), > ChunkSize, > binary_to_list(Format), > binary_to_list(SubChunk1ID), > SubChunk1Size, > AudioFormat, > NumChannels, > SampleRate, > ByteRate, > BlockAlign, > BitsPerSample, > binary_to_list(SubChunk2ID), > SubChunk2Size > ]), > > Binary = < ChunkSize:4/little-unsigned-integer-unit:8, %% > Format:4/binary, > SubChunk1ID:4/binary, > SubChunk1Size:4/little-unsigned-integer-unit:8, > AudioFormat:2/little-unsigned-integer-unit:8, > NumChannels:2/little-unsigned-integer-unit:8, > SampleRate:4/little-unsigned-integer-unit:8, > ByteRate:4/little-unsigned-integer-unit:8, > BlockAlign:2/little-unsigned-integer-unit:8, > BitsPerSample:2/little-unsigned-integer-unit:8, > SubChunk2ID:4/binary, > SubChunk2Size:4/little-unsigned-integer-unit:8, %% > Data/binary > >>, > > file:write_file(FileName, Binary), > {ok, FileName}. > > > > % ----- wave.erl > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From zabrane3@REDACTED Mon May 31 13:29:56 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 31 May 2010 13:29:56 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: 2010/5/31 Gleb Peregud > Try using "lsof" with the process ID of the beam to see if these files > are really closed properly > > $ lsof -c beam.smp doesn't seem to report anything strange (I also tried with beam's PID "lsof -p 17588"). I forget to mention that my code in single threaded, and I only use one process at a time. From max.lapshin@REDACTED Mon May 31 13:37:47 2010 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 31 May 2010 15:37:47 +0400 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C039D91.6050402@erlang-solutions.com> References: <4C03976B.4050501@llaisdy.com> <4C039D91.6050402@erlang-solutions.com> Message-ID: 2010/5/31 Mazen Harake : > Perhaps you should always put a license on it... > By the way:what will happen if not to do it? From gleber.p@REDACTED Mon May 31 13:48:17 2010 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 31 May 2010 13:48:17 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: Does these show anything weird? erlang:ports(). erlang:processes(). erlang:memory(). On Mon, May 31, 2010 at 13:29, zabrane Mikael wrote: > 2010/5/31 Gleb Peregud >> >> Try using "lsof" with the process ID of the beam to see if these files >> are really closed properly >> > > $ lsof -c beam.smp > doesn't seem to report anything strange (I also tried with beam's PID "lsof > -p 17588"). > I forget to mention that my code in single threaded, and I only use one > process at a time. From ivan@REDACTED Mon May 31 14:04:37 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 31 May 2010 13:04:37 +0100 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C039D91.6050402@erlang-solutions.com> References: <4C03976B.4050501@llaisdy.com> <4C039D91.6050402@erlang-solutions.com> Message-ID: <4C03A5D5.4070109@llaisdy.com> Dear Mazen Thank you for your comment. Mazen Harake wrote: > Perhaps you should always put a license on it... I thought a license would be a bit pretentious. What licenses go down well with the Erlang community? Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From ivan@REDACTED Mon May 31 14:11:01 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 31 May 2010 13:11:01 +0100 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: References: <4C03976B.4050501@llaisdy.com> Message-ID: <4C03A755.3040405@llaisdy.com> Dear Max Thank you for your comment. Max Lapshin wrote: > So, if file is 400 MB size, it will return 400MB binary? > > Much more convenient is such protocol: > > wave:pread(File, Offset) -> {ok, header, Header} | {ok, frame, Frame} | eof > > Maybe so? > I suppose it depends on the use cases one has in mind. I'll be working with small files --- less than 1MB --- and I'll want to process the whole file, so reading it all in in a oner seemed sensible. wave:pread(File, 0) would just parse and return the header? That sounds handy. I can't see the use of looking at a single frame; but something like wave:pread(File, Offset, NFrames) could be useful, to read large files in bite-sized chunks. I'll put these down as feature requests :) Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From ulf.wiger@REDACTED Mon May 31 14:19:50 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 31 May 2010 14:19:50 +0200 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C03A5D5.4070109@llaisdy.com> References: <4C03976B.4050501@llaisdy.com> <4C039D91.6050402@erlang-solutions.com> <4C03A5D5.4070109@llaisdy.com> Message-ID: <4C03A966.8070200@erlang-solutions.com> Ivan Uemlianin wrote: > Dear Mazen > > Thank you for your comment. > > Mazen Harake wrote: >> Perhaps you should always put a license on it... > I thought a license would be a bit pretentious. What licenses go down > well with the Erlang community? Any OSI-approved license will do, as long as you're comfortable with it. The Erlang Public License works too, of course. If you want something that is simple and permissive, the ISC license is pretty good. http://en.wikipedia.org/wiki/ISC_license = = = = = Copyright (c) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. = = = = = = BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From alex.arnon@REDACTED Mon May 31 14:33:34 2010 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 31 May 2010 15:33:34 +0300 Subject: [erlang-questions] Erlang-related banners/badges for your site In-Reply-To: References: Message-ID: Can we have that in Russian? :) On 5/31/10, Dmitrii Dimandt wrote: > Hi, all > > If you ever wanted an Erlang-related banner//badge for your site that would > read something like "Erlang FTW!!!", you can now have one: > > http://banners.erlanger.ru/ > > Three sizes, two themes, customized labels. > > I'm open to suggestions (especially with regard to fonts, I have a problem > rendering them correctly on Ubuntu 8.04 Server with Python's PIL) > > > P.S. Since many confuse erlanger with enlarger, Russian Erlang community now > has a sort of meme: "erlang your code" :) > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From zabrane3@REDACTED Mon May 31 14:33:58 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 31 May 2010 14:33:58 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: You're right about your assumption!!! erlang:ports(). -> grows infinitely erlang:processes(). -> are stable (always the same = 69) erlang:memory(). -> memory is increasing So the bug seems to be in another place. How can I be sure that my calls to "port_close" effectively closed the ports? 2010/5/31 Gleb Peregud > Does these show anything weird? > > erlang:ports(). > > erlang:processes(). > > erlang:memory(). > > On Mon, May 31, 2010 at 13:29, zabrane Mikael wrote: > > 2010/5/31 Gleb Peregud > >> > >> Try using "lsof" with the process ID of the beam to see if these files > >> are really closed properly > >> > > > > $ lsof -c beam.smp > > doesn't seem to report anything strange (I also tried with beam's PID > "lsof > > -p 17588"). > > I forget to mention that my code in single threaded, and I only use one > > process at a time. > -- Regards Zabrane From zabrane3@REDACTED Mon May 31 14:54:02 2010 From: zabrane3@REDACTED (zabrane Mikael) Date: Mon, 31 May 2010 14:54:02 +0200 Subject: [erlang-questions] Erlang "system_limit" trouble! In-Reply-To: References: Message-ID: I found the bug (thanks to erlang:port_info/1). ===== PortInfo: [{name,"zlib_drv"}, {links,[<0.2.0>]}, {id,2020}, {connected,<0.2.0>}, {input,0}, {output,26401}] ===== PortInfo: [{name,"zlib_drv"}, {links,[<0.2.0>]}, {id,2021}, {connected,<0.2.0>}, {input,0}, {output,585}] .... The problem was with my wrong usage of "zlib" module. The only place i've use it, is here in this function which try to "uncompress" a GZIP compresses data: uncompress_if_gzip(Bin0) -> try zlib:gunzip(Bin0) of Bin1 -> Bin1 catch _:_ -> % NOT GZIP compressed Bin0 end, Could someone please clarify how to correctly "TRY to uncompress" a binary with "zlib" module? Thousands thanks "Gleb" ... you saved my day ! 2010/5/31 zabrane Mikael > You're right about your assumption!!! > > erlang:ports(). -> grows infinitely > erlang:processes(). -> are stable (always the same = 69) > erlang:memory(). -> memory is increasing > > So the bug seems to be in another place. > How can I be sure that my calls to "port_close" effectively closed > the ports? > > > 2010/5/31 Gleb Peregud > >> Does these show anything weird? >> >> erlang:ports(). >> >> erlang:processes(). >> >> erlang:memory(). >> >> On Mon, May 31, 2010 at 13:29, zabrane Mikael wrote: >> > 2010/5/31 Gleb Peregud >> >> >> >> Try using "lsof" with the process ID of the beam to see if these files >> >> are really closed properly >> >> >> > >> > $ lsof -c beam.smp >> > doesn't seem to report anything strange (I also tried with beam's PID >> "lsof >> > -p 17588"). >> > I forget to mention that my code in single threaded, and I only use one >> > process at a time. >> > > > > -- > Regards > Zabrane > -- Regards Zabrane From kenji.rikitake@REDACTED Mon May 31 15:17:32 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Mon, 31 May 2010 22:17:32 +0900 Subject: [erlang-questions] Speedy unsort:shuffle/1,2 ? In-Reply-To: References: <4BFD16EF.6060206@erix.ericsson.se> <6AE7BE26-D21C-4DBD-914F-EE2067B08128@telegraphics.com.au> <5DD3CDF1-6148-46FC-8496-A3010B97BD30@cs.otago.ac.nz> <20100530075320.GA29981@k2r.org> Message-ID: <20100531131732.GA63905@k2r.org> In the message dated Mon, May 31, 2010 at 01:08:29PM +1200, Richard O'Keefe writes: > On May 30, 2010, at 7:53 PM, Kenji Rikitake wrote: > >For a testing purpose, the RNG doesn't really have to be > >cryptographycally safe; it only has to have a uniform distribution. > > Where did "cryptographically safe" come from? Mea Culpa. I mixed up the context - some articles in the thread mentioned the RNG in crypto module. > By better, I meant > - faster > - having better statistical properties Agreed. > >Anybody has implemented a NIF or port version of Mersenne Twister or > >SFMT? > >http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index.html > > and those are excellent examples of what I meant. Understood. Kenji Rikitake From mazen.harake@REDACTED Mon May 31 16:43:24 2010 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 31 May 2010 17:43:24 +0300 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C03A5D5.4070109@llaisdy.com> References: <4C03976B.4050501@llaisdy.com> <4C039D91.6050402@erlang-solutions.com> <4C03A5D5.4070109@llaisdy.com> Message-ID: <4C03CB0C.3010807@erlang-solutions.com> It's better to be safe than sorry; always put a license on things you expect people to use... otherwise they won't ;) I say this because I have seen my share of "I found this old code in this thread written x years ago... anyone know the license of the code?". :) Personally I've been a fan of licenses that let you do almost what ever you want (I prefer BSD licenses) but as Ulf said you can go with any which is OSI approved. /Mazen On 31/05/2010 15:04, Ivan Uemlianin wrote: > Dear Mazen > > Thank you for your comment. > > Mazen Harake wrote: >> Perhaps you should always put a license on it... > I thought a license would be a bit pretentious. What licenses go down > well with the Erlang community? > Best wishes > > Ivan > --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From ivan@REDACTED Mon May 31 16:53:09 2010 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 31 May 2010 15:53:09 +0100 Subject: [erlang-questions] Reading and Writing .wav files In-Reply-To: <4C03CB0C.3010807@erlang-solutions.com> References: <4C03976B.4050501@llaisdy.com> <4C039D91.6050402@erlang-solutions.com> <4C03A5D5.4070109@llaisdy.com> <4C03CB0C.3010807@erlang-solutions.com> Message-ID: <4C03CD55.1020804@llaisdy.com> Mazen, Ulf Thanks for your comments. I'll repost (here and on my blog) with an OSI license attached. Best Ivan Mazen Harake wrote: > It's better to be safe than sorry; always put a license on things you > expect people to use... otherwise they won't ;) > > I say this because I have seen my share of "I found this old code in > this thread written x years ago... anyone know the license of the > code?". :) > > Personally I've been a fan of licenses that let you do almost what > ever you want (I prefer BSD licenses) but as Ulf said you can go with > any which is OSI approved. > > /Mazen -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From fritchie@REDACTED Mon May 31 22:10:59 2010 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 31 May 2010 15:10:59 -0500 Subject: [erlang-questions] generic replication bahaviour In-Reply-To: Message of "Wed, 26 May 2010 12:31:29 +0200." Message-ID: <39655.1275336659@snookles.snookles.com> Replying to a thead from last week.... Johan Montelius wrote: jm> has anyone played around with a generic replication behaviour in the jm> style of gen_server? No, sorry. But before doing such a thing, I think you need to put some thought into what "replication" means vis a vis Brewer's CAP theorem. Or more perhaps more usefully(*), the spectrum of consistency vs. availability. Then (perhaps?) deciding on an implementation that's based on a state machine replication technique or a quorum-based technique. The choice of technique may help drive what kind of metadata you're going to require for each thingie stored. Are the thingie's key-value pairs, or something else? Do you require monotonically-increasing timestamps or vector clocks or something else? Does the behavior always resolve consistency ambiguity or push those decisions to the client, and how the client app inform the behavior of its choice? If you're dealing with disk-based persistence, then any Erlang process that's doing disk I/O can block at very inconvenient times for very inconvenient lengths of time. Syncronous replication across a network (even a fast LAN) can result in similiar inconveniences, at least in terms of Murphy's Law. The sum of these inconveniences can easily tip an implementation into the "Won't work for my app" category. Just things off the top of the cuff of my head. :-) I'll make a blind guess and say that this is why key-value stores such as Dynomite, Scalaris, Riak, Ringo, Scalien, and others are already "out there"(**) and useful: they choose a path through the maze of choices above and then do it well. -Scott (*) Reading Brewer's writings about CAP and then the Gilbert & Lynch proof, the formal definition of "P" is a tricky thing. (**) Sorry, Hibari hasn't been released yet, contrary to what my Erlang Factory 2010 San Francisco talk had predicted. Mid-July 2010 is my best guess right now.