From gnoblin@REDACTED Tue Mar 1 01:03:33 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Tue, 1 Mar 2011 02:03:33 +0200 Subject: launch .exe files on Windows via Erlang Message-ID: Hello! Excuse me for probably an extremely lame question: how can I execute .exe file on windows with certain command line flag via Erlang? thanks, Slav From mapandfold@REDACTED Tue Mar 1 03:48:15 2011 From: mapandfold@REDACTED (MapAndFold) Date: Tue, 1 Mar 2011 10:48:15 +0800 Subject: [erlang-questions] Success Stories of Erlang Development In-Reply-To: <1298919606.6407.8.camel@marcosluis-Aspire-5251> References: <1298919606.6407.8.camel@marcosluis-Aspire-5251> Message-ID: Hi, Is Google Talk using Erlang? Any credit for this? On Tue, Mar 1, 2011 at 3:00 AM, Marcos Ortiz wrote: > Regards to all the list > I?m searching several projects that have had success because they have > used Erlang on its core. > > I know cases like Facebook (its chat platform), Google (Google Talk) but > I don?t know more. > > Can you give some links to review the state-of-the-art today? > > If you can give more examples resolved (not only chat apps), It would be > awesome. > > Thanks a lot for your time > -- > ?Marcos Lu?s Ort?z Valmaseda > ?Software Engineer > ?Centro de Tecnolog?as de Gesti?n de Datos (DATEC) > ?Universidad de las Ciencias Inform?ticas > ?http://uncubanitolinuxero.blogspot.com > ?http://www.linkedin.com/in/marcosluis2186 > > From mlortiz@REDACTED Tue Mar 1 03:52:13 2011 From: mlortiz@REDACTED (Marcos Ortiz Valmaseda) Date: Mon, 28 Feb 2011 21:52:13 -0500 (CST) Subject: [erlang-questions] Success Stories of Erlang Development In-Reply-To: Message-ID: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> Sorry, my mistake. Google Talk is based on XMPP. Well, at least, the Wrox Professional XMPP Development with Javascript and JQuery says that. ----- Mensaje original ----- De: "MapAndFold" Para: "Marcos Ortiz" CC: erlang-questions@REDACTED Enviados: Lunes, 28 de Febrero 2011 21:48:15 (GMT-0500) Auto-Detected Asunto: Re: [erlang-questions] Success Stories of Erlang Development Hi, Is Google Talk using Erlang? Any credit for this? On Tue, Mar 1, 2011 at 3:00 AM, Marcos Ortiz wrote: > Regards to all the list > I?m searching several projects that have had success because they have > used Erlang on its core. > > I know cases like Facebook (its chat platform), Google (Google Talk) but > I don?t know more. > > Can you give some links to review the state-of-the-art today? > > If you can give more examples resolved (not only chat apps), It would be > awesome. > > Thanks a lot for your time > -- > ?Marcos Lu?s Ort?z Valmaseda > ?Software Engineer > ?Centro de Tecnolog?as de Gesti?n de Datos (DATEC) > ?Universidad de las Ciencias Inform?ticas > ?http://uncubanitolinuxero.blogspot.com > ?http://www.linkedin.com/in/marcosluis2186 > > -- Marcos Lu?s Ort?z Valmaseda Software Engineer Universidad de las Ciencias Inform?ticas Linux User # 418229 http://uncubanitolinuxero.blogspot.com http://www.linkedin.com/in/marcosluis2186 From ok@REDACTED Tue Mar 1 04:10:24 2011 From: ok@REDACTED (ok@REDACTED) Date: Tue, 1 Mar 2011 16:10:24 +1300 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: References: <4D6BE311.6030903@cs.ntua.gr> Message-ID: <1aa65ed5695edd0263e9bfb6b552236d.squirrel@chasm.otago.ac.nz> > Because that's not how his original program was flowing. His use of the C > "return" keyword indicated to me that the flow was important. That there > was a particular type of flow he couldn't figure out how to do, and *that* > was what he was interested in. Not the function itself. Yes, but the multiclause function does NOT mimic the flow of the original bellower. There _is_ a simple straightforward way to mimic arbitrarily positioned use of 'return' in Erlang: wrap the body of the function in a try..catch and replace 'return' by throw. That's pretty much a perfect mimic, but it's not a good way to write the code. It's almost NEVER a good idea to take the flow of a C function and try to mimic it in Erlang. It is necessary to learn how to use Erlang effectively, and then go back to what the function is supposed to mean and restart writing new code in Erlang. (There is of course a well known way to translate any arbitrary flow chart into pure functional code. But you don't want to go there either.) > From mapandfold@REDACTED Tue Mar 1 04:16:28 2011 From: mapandfold@REDACTED (MapAndFold) Date: Tue, 1 Mar 2011 11:16:28 +0800 Subject: [erlang-questions] launch .exe files on Windows via Erlang In-Reply-To: References: Message-ID: Hi, For example, os:cmd("werl.exe -detached"). Take care when you are executing an exe with full file name. It's painful if my memory is correct and I have changed working dir with file:set_cwd/1 before execution. On Tue, Mar 1, 2011 at 8:03 AM, Slav Pankratov wrote: > Hello! > > Excuse me for probably an extremely lame question: > > how can I execute .exe file on windows with certain command line flag via > Erlang? > > thanks, > Slav > -- Best Regards, Zhengji From ok@REDACTED Tue Mar 1 04:27:58 2011 From: ok@REDACTED (ok@REDACTED) Date: Tue, 1 Mar 2011 16:27:58 +1300 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: <20110228125820.GA27813@hanele.lan> References: <20110228125820.GA27813@hanele.lan> Message-ID: >> > And one more option: > > judge() -> > judge(get_a(), get_b()). > > judge(A, _) when A == conf1 -> > do_some_thing(A); > judge(_, B) when B == conf2 -> > do_other_thing(B); > judge(_, _) -> > do_things(). This changes the control flow. We have no reason to believe that it is even safe to call get_b() when get_a() is CONF1. From jack@REDACTED Tue Mar 1 04:31:44 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 28 Feb 2011 20:31:44 -0700 Subject: [erlang-questions] Success Stories of Erlang Development In-Reply-To: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> References: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> Message-ID: > Sorry, my mistake. Google Talk is based on XMPP. Well, at least, the Wrox Professional XMPP Development with Javascript and JQuery says that. I believe they implemented their own XMPP stack in Java (or perhaps C++). Erlang is, unfortunately, not one of their sanctioned languages. My previous startups Chesspark and Collecta both used Erlang. At Collecta, we used it for nearly everything we did (the next most used language was JavaScript). I believe there is a video of a talk I gave at last year's Erlang Factory about how it was used at Collecta. One of the engineers I managed at Collecta is now at Mochi Media, where Erlang is also used quite heavily (and successfully). Github uses Erlang for various things too. You should really consider attending one of the Erlang Factories. Most of the attendees are using Erlang in their projects for something really cool. jack. From mapandfold@REDACTED Tue Mar 1 04:47:23 2011 From: mapandfold@REDACTED (MapAndFold) Date: Tue, 1 Mar 2011 11:47:23 +0800 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: References: Message-ID: Hi, I like such generalization. But I don't think it is general and clean enough. My suggestion is composewhile. (The term "Compose" is from Mathematica) composewhile([], Acc) -> Acc; composewhile([F | T], Acc) -> case F(Acc) of {ok, Acc0} -> composewhile(T, Acc0); {stop, Acc0} -> Acc0 end. And we will arrange our job into steps like this: stepA(Acc) -> A = getA(), case A == ?CONF1 of true -> doSomeThing(A), {ok, Acc}; _ -> {stop, Acc} end. stepC(Acc) -> doThings(), {ok, Acc}. All composed together: composewhile([fun stepA/1, fun stepB/1, fun stepC/1], 0). 2011/2/28 ??? (Yau-Hsien Huang) : > > > ? ?test_seq( State, [] ) -> State ; > ? ?test_seq( State, [{S,O,F}|SOFs] ) -> > ? ? ? ?case compare(S, O, F) of > ? ? ? ? ? ?nil -> test_seq(State, SOFs) ; > ? ? ? ? ? ?Result -> manage(State, apply, F, [S]) > ? ? ? ?end. > > that manage/N takes care of State. > > -- > Y.-H. H. > -- Best Regards, mapandfold From alexey.v.romanov@REDACTED Tue Mar 1 08:03:44 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 1 Mar 2011 10:03:44 +0300 Subject: [erlang-questions] how do people get edoc to coexist with dialyzer? In-Reply-To: <19819.43019.965963.268924@ornendil.otp.ericsson.se> References: <20110228121116.GA5848@corelatus.se> <19819.43019.965963.268924@ornendil.otp.ericsson.se> Message-ID: On Mon, Feb 28, 2011 at 4:50 PM, Hans Bolinder wrote: > issues, and the exact behaviour will change a bit. Nevertheless we > expect to release an updated EDoc in the upcoming R14B02 release. That's very good news! Yours, Alexey Romanov From per@REDACTED Tue Mar 1 08:31:34 2011 From: per@REDACTED (Per Hedeland) Date: Tue, 1 Mar 2011 08:31:34 +0100 (CET) Subject: [erlang-questions] crypto NIF vs OpenSSL problems In-Reply-To: <4D6BDA53.8010807@erix.ericsson.se> Message-ID: <201103010731.p217VYjt028258@pluto.hedeland.org> Sverker Eriksson wrote: > >Per Hedeland wrote: >> Hi, >> >> Pre-R14, if libcrypto was missing / wrong version, >> application:start(crypto) would return a nice error (well, at least an >> error). crypto:start() would return the same error. >> >> With the same scenario in R14, application:start(crypto) succeeds(!), >> and crypto:start() crashes with undef. I understand completely why on >> both counts, but, um... - is this intentional? Or a bug (or two) that >> will be fixed >Maybe you could say that application:start(crypto) succeeding is a bug >and it should fail with nice error as before. Yes, at least some failure indication would be appropriate I think - it's rather weird to have application:start() succeed even though *any* use of the application will fail. Of course (with the current crypto design) this can be easily achieved by having application:start/0 attempt to load the crypto module - e.g. a silly call of crypto:version/0 from crypto_server:init/1 does the trick. >I don't think I want to revert the behavior of crypto:start() as that is >"clearly undocumented" and undef is what you get when trying to call a >function in module that can not be loaded. Yes, you're probably right - undef is kind of "logical" (and it's certainly true that the module couldn't be loaded). And now you have the nice (though also undocumented:-) feature that you can just call any function in crypto without bothering with either type of start - it would be non-backwards-compat to lose that.:-) Hm, thinking about a way to do the latter without the use of on_load - would it be useful to have some "magic" to let a NIF stub work a la error_handler:undefined_function/3? You could of course do it "manually" with something like md4(Data) -> load_the_nifs(md4, [Data]). md4_init() -> load_the_nifs(md4_init, []). and so on, but this would obviously be both boring and error-prone. --Per From lukas.larsson@REDACTED Tue Mar 1 09:26:50 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Tue, 1 Mar 2011 08:26:50 +0000 (GMT) Subject: [erlang-questions] Dialyzer HttpPacket Waring (bug?) In-Reply-To: Message-ID: <907120227.3091298968010914.JavaMail.root@zimbra> I thought I fixed this bug in R14B01, see https://github.com/erlang/otp/blob/fa6d9d530ac55dd47eb10fde181084d3886ee83c/lib/hipe/cerl/erl_bif_types.erl for the commit which does it. Are you sure that you are running R14B01? Lukas ----- Original Message ----- From: "Edmond Begumisa" To: "Kostis Sagonas" Cc: erlang-questions@REDACTED Sent: Monday, 28 February, 2011 22:14:13 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [erlang-questions] Dialyzer HttpPacket Waring (bug?) On Tue, 01 Mar 2011 07:40:34 +1100, Kostis Sagonas wrote: > Edmond Begumisa wrote: >> On Tue, 01 Mar 2011 07:10:44 +1100, Edmond Begumisa >> wrote: >> >>> Hello all, >>> >>> I'm using erlang:decode_packet/3 to do some HTTP. When I use the >>> http_bin packet type and match the result, Dialyzer keeps warning >>> about my HttpString type being invalid indicating that it's expecting >>> a string() and not my binary(). Yet the documentation says (and >>> tracing confirms) that HttpString = string() | binary() >>> >>> For example... {SNIP} > > Please learn how to submit a proper bug report. Which OTP version is > this? Which documentation you are looking at? Oops, forgive me. Not sleeping much :) OTP: R14B01 DOC: www.erlang.org/doc/ OS: Win7 + Ubuntu 9.1 Kernel 2.6.31-14-generic - Edmond - > I cannot reproduce it with R14B01. > Upgrade to it and your problem will go away. > > Kostis > Sorry, > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From ingela@REDACTED Tue Mar 1 09:48:51 2011 From: ingela@REDACTED (Ingela Andin) Date: Tue, 1 Mar 2011 09:48:51 +0100 Subject: [erlang-questions] public_key and ssl question In-Reply-To: <1298910042.19215.34.camel@ws-aschultz> References: <1298884906.19215.20.camel@ws-aschultz> <1298889048.19215.28.camel@ws-aschultz> <1298910042.19215.34.camel@ws-aschultz> Message-ID: Hi! No oids are not translated into atoms, we use the asn1 generated macros instead in the code. Ex: #'AttributeTypeAndValue'{type=?'id-emailAddress', value=Mail} It would be a bad idea to translate oids to atoms as there is no gurantee that they will be uinque. Regards Ingela Erlang OTP team - Ericsson AB 2011/2/28 Andreas Schultz : > Hi Ingla, > > I think i understand the decoding a bit better now. I guess i was mostly > confused by getting something like: > > #'AttributeTypeAndValue'{type = {2,5,4,3}, > ? ? ? ? ? ? ? ? ? ? ? ? value = {teletexString,"www.example.com"}} > > when is was somehow expecting something more like: > > #'AttributeTypeAndValue'{type = 'X520OrganizationName', > ? ? ? ? ? ? ? ? ? ? ? ? value = {teletexString,"www.example.com"}}, > > Or in other words, i was expecting the numeric ASN.1 identifiers to be > decoded into atom. But it seems that is not the way the ASN.1 stuff in > Erlang works. > > So, this leaves on the missing commonName validation if the ssl app > works as client. > > Andreas > > On Mon, 2011-02-28 at 12:21 +0100, Ingela Andin wrote: >> Hi again, >> >> I can not really answer this from the top of my head ( at home taking >> care of sick kids today), >> I will check the specs when I get back, if we are indeed missing >> something which ofcourse >> is possible, we would be interested in your patch suggestion. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> 2011/2/28 Andreas Schultz : >> > Hi, >> > >> > On Mon, 2011-02-28 at 11:16 +0100, Ingela Andin wrote: >> >> Hi! >> >> >> >> 2011/2/28 Andreas Schultz : >> >> > Hi, >> >> > >> >> > I have been playing with the public_key, crypto and ssl apps a bit. >> >> > >> >> > What struck me as odd is that X.509 certificates are only partially >> >> > decoded. public_key:pkix_decode_cert/2 will return many of the fields >> >> > undecoded. This leads to my first question: >> >> > >> >> > ?Is the partial decoding on purpose (maybe to simplify things or safe >> >> > ?memory/cpu cycle) or just not completed yet? >> >> >> >> There are two ways of decoding a cert, plain and otp. Plain uses the >> >> standard ASN1-spec and only decodes the top level (Default Asn1 >> >> application behaviour). ?You may use public_key:der_decode/2 to look >> >> at parts. ?If you use otp-decoding much more will be decoded, but >> >> there still might be non standard extensions that will not be decoded. >> >> (Then you may need your own ASN1-spec) >> > >> > Yeah, i have seen the difference between otp and plain decoding and i >> > was actually referring to the otp decoding. Many to the standard PKCS1 >> > parts of a certificate are not decoded. For example most (if not all) >> > parts of a typical subject and decode only as printableString. The >> > subject from a valid Thawte certificate will look like this: >> > >> > ? ? ? ? ? ?subject = >> > ? ? ? ? ? ? ? ?{rdnSequence, >> > ? ? ? ? ? ? ? ? ? ?[[#'AttributeTypeAndValue'{ >> > ? ? ? ? ? ? ? ? ? ? ? ? ?type = {2,5,4,10}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ?value = {teletexString,"www.example.com"}}], >> > ? ? ? ? ? ? ? ? ? ? [#'AttributeTypeAndValue'{ >> > ? ? ? ? ? ? ? ? ? ? ? ? ?type = {2,5,4,11}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ?value = >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{printableString, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"Go to >> > https://www.thawte.com/repository/index.html"}}], >> > ? ? ? ? ? ? ? ? ? ? [#'AttributeTypeAndValue'{ >> > ? ? ? ? ? ? ? ? ? ? ? ? ?type = {2,5,4,11}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ?value = {printableString,"Thawte SSL123 >> > certificate"}}], >> > ? ? ? ? ? ? ? ? ? ? [#'AttributeTypeAndValue'{ >> > ? ? ? ? ? ? ? ? ? ? ? ? ?type = {2,5,4,11}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ?value = {printableString,"Domain >> > Validated"}}], >> > ? ? ? ? ? ? ? ? ? ? [#'AttributeTypeAndValue'{ >> > ? ? ? ? ? ? ? ? ? ? ? ? ?type = {2,5,4,3}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ?value = {teletexString,"www.example.com"}}]]}, >> > >> > Most the subject parts will never be used. But at least the common name >> > would be interesting. Yet, i believe that if the subject had an >> > alternateEmail attribute, that would be decoded.... >> > >> > Anyway, the real question is, would it be worthwhile to add the missing >> > bits to the OTP ASN.1 module and submit a patch? >> > >> >> > And second: >> >> > >> >> > ?Since the common name (CN) of a certificate is not decoded, how will >> >> > ?a SSL client verify the identity of a SSL server? >> >> > >> >> > Looking through the SSL code i can see that the certificate chain itself >> >> > will be verified (the verify_peer result), but the common name is not >> >> > check as far as i can tell. >> >> > >> >> >> >> Ssl uses the otp-decoding and the public_key application to do a >> >> path-validation according to RFC-5280. ( We have not yet ?finished the >> >> implementation of CRL handling, but except for those all mandantory >> >> parts should be in place). >> > >> > The path-validation yes, but the most basic validation if the >> > certificate even belongs to the host we think we are talking to seems to >> > be missing. Or am i missing something? >> > >> > Regards >> > Andreas >> > >> >> >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> > >> > -- >> > Dipl. Inform. >> > Andreas Schultz >> > >> > email: as@REDACTED >> > phone: +49-391-819099-224 >> > mobil: +49-170-2226073 >> > >> > ------------------ managed broadband access ------------------ >> > >> > Travelping GmbH ? ? ? ? ? ? ? phone: ? ? ? ? ? +49-391-8190990 >> > Roentgenstr. 13 ? ? ? ? ? ? ? fax: ? ? ? ? ? +49-391-819099299 >> > D-39108 Magdeburg ? ? ? ? ? ? email: ? ? ? info@REDACTED >> > GERMANY ? ? ? ? ? ? ? ? ? ? ? web: ? http://www.travelping.com >> > >> > Company Registration: Amtsgericht Stendal Reg No.: ? HRB 10578 >> > Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 >> > -------------------------------------------------------------- >> > > > -- > Dipl. Inform. > Andreas Schultz > > email: as@REDACTED > phone: +49-391-819099-224 > mobil: +49-170-2226073 > > ------------------ managed broadband access ------------------ > > Travelping GmbH ? ? ? ? ? ? ? phone: ? ? ? ? ? +49-391-8190990 > Roentgenstr. 13 ? ? ? ? ? ? ? fax: ? ? ? ? ? +49-391-819099299 > D-39108 Magdeburg ? ? ? ? ? ? email: ? ? ? info@REDACTED > GERMANY ? ? ? ? ? ? ? ? ? ? ? web: ? http://www.travelping.com > > Company Registration: Amtsgericht Stendal Reg No.: ? HRB 10578 > Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 > -------------------------------------------------------------- > From ebegumisa@REDACTED Tue Mar 1 11:10:10 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 01 Mar 2011 21:10:10 +1100 Subject: [erlang-questions] Dialyzer HttpPacket Waring (bug?) In-Reply-To: <907120227.3091298968010914.JavaMail.root@zimbra> References: <907120227.3091298968010914.JavaMail.root@zimbra> Message-ID: Actually, you are both correct. My IDE was pre-loading CouchDB which comes with it's own older OTP. So when dialyzing from my IDE, I was actually running in an older OTP with the bug rather than the latest one I normally work with. I only realised this when I noticed the bug would only appear in my applications that use CouchDB. I've rectified this and everything is fine. Thanks Lukas and Kostis for your help. - Edmond - On Tue, 01 Mar 2011 19:26:50 +1100, Lukas Larsson wrote: > I thought I fixed this bug in R14B01, see > https://github.com/erlang/otp/blob/fa6d9d530ac55dd47eb10fde181084d3886ee83c/lib/hipe/cerl/erl_bif_types.erl > for the commit which does it. Are you sure that you are running R14B01? > > Lukas > ----- Original Message ----- > From: "Edmond Begumisa" > To: "Kostis Sagonas" > Cc: erlang-questions@REDACTED > Sent: Monday, 28 February, 2011 22:14:13 GMT +01:00 Amsterdam / Berlin / > Bern / Rome / Stockholm / Vienna > Subject: Re: [erlang-questions] Dialyzer HttpPacket Waring (bug?) > > On Tue, 01 Mar 2011 07:40:34 +1100, Kostis Sagonas > wrote: > >> Edmond Begumisa wrote: >>> On Tue, 01 Mar 2011 07:10:44 +1100, Edmond Begumisa >>> wrote: >>> >>>> Hello all, >>>> >>>> I'm using erlang:decode_packet/3 to do some HTTP. When I use the >>>> http_bin packet type and match the result, Dialyzer keeps warning >>>> about my HttpString type being invalid indicating that it's expecting >>>> a string() and not my binary(). Yet the documentation says (and >>>> tracing confirms) that HttpString = string() | binary() >>>> >>>> For example... {SNIP} >> >> Please learn how to submit a proper bug report. Which OTP version is >> this? Which documentation you are looking at? > > Oops, forgive me. Not sleeping much :) > > OTP: R14B01 > DOC: www.erlang.org/doc/ > OS: Win7 + Ubuntu 9.1 Kernel 2.6.31-14-generic > > - Edmond - > >> I cannot reproduce it with R14B01. >> Upgrade to it and your problem will go away. >> >> Kostis >> > > Sorry, > > > >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From robert.virding@REDACTED Tue Mar 1 12:01:41 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 1 Mar 2011 11:01:41 +0000 (GMT) Subject: [erlang-questions] bit syntax: what does "<<1234>> =" match? In-Reply-To: <944045824.4141298976998712.JavaMail.root@zimbra> Message-ID: <1897964628.4311298977301309.JavaMail.root@zimbra> ----- "Anthony Ramine" wrote: > So we agree, but I misread what you said and thought the opposite ;) > The compiler does not complain with <<1234>> and that's already weird > as it is. It gets to a whole new level of weirdness when you discover > that <> is also a legal pattern even though it won't ever match. Yes, I checked the compiler as well and, unfortunately :-), you are right. If you check the generated code you will find that works in the same way as the interpreter: it first extracts the first 8 bit integer (default as no size or type specified) and then tries to match it against the atom foo, which fails. I would have thought that erl_lint should have caught this. Just to boast a bit: in LFE this results in an error in both the compiler and interpreter. :-) Robert -- Robert Virding, Erlang Solutions Ltd. From alain.odea@REDACTED Tue Mar 1 14:52:39 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 1 Mar 2011 10:22:39 -0330 Subject: [erlang-questions] Re: Success Stories of Erlang Development In-Reply-To: <1298921946.6407.13.camel@marcosluis-Aspire-5251> References: <1298919606.6407.8.camel@marcosluis-Aspire-5251> <6206E9909D384EDB8171E5649D22280B@srstrong.com> <1298921946.6407.13.camel@marcosluis-Aspire-5251> Message-ID: <44F4052E-95E5-483A-B9C1-D2E04CF12898@gmail.com> If you are using Django you may find Zotonic compelling. I built http://verafin.com/ using it and Verafin was very pleased with the speed of development and the fast page loads of the site itself even though it is hosted on a low-end virtual private server. I can only imagine the site on dedicated hardware... On 2011-02-28, at 16:09, Marcos Ortiz wrote: > Thanks a lot to everyone for the quick response. > We are planning to use Erlang for several projects combined with other > technologies like Django, Ruby on Rails, Apache Hadoop, etc. > > When I had everything in my hands, I will share on my blog all stuff. > > Thanks again > -- > Marcos Lu?s Ort?z Valmaseda (@marcosluis2186) > Software Engineer > Centro de Tecnolog?as de Gesti?n de Datos (DATEC) > Universidad de las Ciencias Inform?ticas > http://uncubanitolinuxero.blogspot.com > http://www.linkedin.com/in/marcosluis2186 > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From francesco@REDACTED Tue Mar 1 14:59:47 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 01 Mar 2011 13:59:47 +0000 Subject: [erlang-questions] Success Stories of Erlang Development In-Reply-To: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> References: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> Message-ID: <4D6CFBD3.8000708@erlang-solutions.com> I would suggest you browse the Erlang Factory website where we have recordings of most events. You will find companies such as Yahoo!, E*Trade, Electronic Arts, Ericsson, Nortel (RIP), Klarna, all the way to start-ups and established SMEs such as Basho, CouchBase, Github, EngineYard, Heroku, Procecss-One, Erlang Solutions, Quviq, Rabbit Technologies (Currently VM Ware) and more, all talking about their success stories. The recordings are currently sorted by conference. We hope to make the index the archive in the near future, organizing the talks by themes and topics. Right now, there are about 150 talks on the site. The URL is http://www.erlang-factory.com/past_conferences Hope this helps, Francesco On 01/03/2011 02:52, Marcos Ortiz Valmaseda wrote: > Sorry, my mistake. Google Talk is based on XMPP. Well, at least, the Wrox Professional XMPP Development with Javascript and JQuery says that. > > ----- Mensaje original ----- > De: "MapAndFold" > Para: "Marcos Ortiz" > CC: erlang-questions@REDACTED > Enviados: Lunes, 28 de Febrero 2011 21:48:15 (GMT-0500) Auto-Detected > Asunto: Re: [erlang-questions] Success Stories of Erlang Development > > Hi, > > Is Google Talk using Erlang? Any credit for this? > > On Tue, Mar 1, 2011 at 3:00 AM, Marcos Ortiz wrote: >> Regards to all the list >> I?m searching several projects that have had success because they have >> used Erlang on its core. >> >> I know cases like Facebook (its chat platform), Google (Google Talk) but >> I don?t know more. >> >> Can you give some links to review the state-of-the-art today? >> >> If you can give more examples resolved (not only chat apps), It would be >> awesome. >> >> Thanks a lot for your time >> -- >> Marcos Lu?s Ort?z Valmaseda >> Software Engineer >> Centro de Tecnolog?as de Gesti?n de Datos (DATEC) >> Universidad de las Ciencias Inform?ticas >> http://uncubanitolinuxero.blogspot.com >> http://www.linkedin.com/in/marcosluis2186 >> >> -- Erlang Solutions Ltd. http://www.erlang-solutions.com From raimo+erlang-questions@REDACTED Tue Mar 1 16:00:16 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 1 Mar 2011 16:00:16 +0100 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: References: Message-ID: <20110301150016.GA18214@erix.ericsson.se> On Mon, Feb 28, 2011 at 11:14:31AM +0800, Wang Wei wrote: > Hello, I has a question about how to convert the bellow C program into Erlang. > > void judge() > { > int a; > int b; > > a = getA(); > if (a == CONF1) > { > doSomeThing(a); > return; > } > > b = getB(); > if (b == CONF2) > { > doOtherThing(b); > return; > } > > doThings(); > return; > } judge() -> A = getA(), if A =:= 'CONF1' -> doSomeThing(A); true -> B = getB(), if B =:= 'CONF2' -> doOtherThing(B); true -> doThings() end end. But I guess that was a too obious transformation that did not answer the question. Provided called functions do not throw/1: judge() -> try A = getA(), if A =:= 'CONF1' -> doSomeThing(A), throw(ok); true -> ok end, B = getB(), if B =:= 'CONF2' -> doOtherThing(B), throw(ok); true -> ok end, doThings(), ok catch Result -> Result end. And if they can throw(Anything) elaborate with: judge() -> OK = make_ref(), : : throw(OK); : : throw(OK); : : catch OK -> ok; Other -> erlang:raise(throw, Other, erlang:get_stacktrace()) end. But only library code should have to be prepared for such nonsense. > > I think about "case" and "if" construct, but none of it seems work fine, thanks for help. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From nox@REDACTED Tue Mar 1 16:12:10 2011 From: nox@REDACTED (Anthony Ramine) Date: Tue, 1 Mar 2011 16:12:10 +0100 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: <20110301150016.GA18214@erix.ericsson.se> References: <20110301150016.GA18214@erix.ericsson.se> Message-ID: Le 1 mars 2011 ? 16:00, Raimo Niskanen a ?crit : > And if they can throw(Anything) elaborate with: > > judge() -> > OK = make_ref(), > : > : > throw(OK); > : > : > throw(OK); > : > : > catch > OK -> ok; > Other -> > erlang:raise(throw, Other, erlang:get_stacktrace()) > end. > > But only library code should have to be prepared for such nonsense. Why catch something to raise it immediately afterwards? -- Anthony Ramine Dev:Extend http://dev-extend.eu From matthias@REDACTED Tue Mar 1 17:33:05 2011 From: matthias@REDACTED (Matthias Lang) Date: Tue, 1 Mar 2011 17:33:05 +0100 Subject: is anyone else getting these one-question mails? Message-ID: <20110301163305.GA3473@corelatus.se> Hi, this morning, someone with a hotmail account sent me mail asking: > Hi,I want to know is Erlang suitable for writing an IDS (Intrusion > Detection System)?Please help me.Best Regards I answered "Yes.". Just a few minutes ago, a different name with a yahoo account sent me mail saying > Hi,Can I use Erlang programming for HPC?Thanks for your??guidance Notice how neither of 'them' use spaces after punctuation. Is anyone else getting these? It feels a bit like a bot. A bot which asks marginally relevant questions about Erlang. How odd. Matt From gleber.p@REDACTED Tue Mar 1 17:39:18 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Tue, 1 Mar 2011 17:39:18 +0100 Subject: [erlang-questions] is anyone else getting these one-question mails? In-Reply-To: <20110301163305.GA3473@corelatus.se> References: <20110301163305.GA3473@corelatus.se> Message-ID: On Tue, Mar 1, 2011 at 17:33, Matthias Lang wrote: > Is anyone else getting these? No > It feels a bit like a bot. Yeah, it feels like bot a bit From raimo+erlang-questions@REDACTED Tue Mar 1 17:56:05 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 1 Mar 2011 17:56:05 +0100 Subject: [erlang-questions] is there "return" in Erlang. In-Reply-To: References: <20110301150016.GA18214@erix.ericsson.se> Message-ID: <20110301165605.GA24660@erix.ericsson.se> On Tue, Mar 01, 2011 at 04:12:10PM +0100, Anthony Ramine wrote: > Le 1 mars 2011 ? 16:00, Raimo Niskanen a ?crit : > > > And if they can throw(Anything) elaborate with: > > > > judge() -> > > OK = make_ref(), > > : > > : > > throw(OK); > > : > > : > > throw(OK); > > : > > : > > catch > > OK -> ok; > > Other -> > > erlang:raise(throw, Other, erlang:get_stacktrace()) > > end. > > > > But only library code should have to be prepared for such nonsense. > > Why catch something to raise it immediately afterwards? That is a re-raise preserving original stacktrace of anything thrown by any called functions. You are right, it was oversmart and redundant. Well spotted. Catching OK -> ok; only is enough. All other exceptions are automatically re-raised as my redundant code does. So a better ending is: : : catch OK -> ok end. > > -- > Anthony Ramine > Dev:Extend > http://dev-extend.eu > > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From comptekki@REDACTED Tue Mar 1 18:46:34 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 1 Mar 2011 10:46:34 -0700 Subject: catch Message-ID: I have this escript to create a list of output: main([String]) -> case length(String) of 0 -> io:format("~nDid you forget the number?~n~n"); _ -> case catch(list_to_integer(String)) of {'EXIT', {badarg,_}} -> io:format("~n~p~n~n",["Arg not Int"]); _ -> I=list_to_integer(String), Seq=lists:seq(I+1, I+20), lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) end end; main(_) -> io:format("~nDid you forget the number?~n~n"). I first tried: case catch(I=list_to_integer(String)) of But I think there was an error that I was unsafe. Is there a way to do a variable assignment to retain a good value so you don't have to do the catch then redo it when it is valid? I.e., list_to_ineger will be run twice if it is a valid integer. Is there a way to do this once? thx, -wes From fernando.benavides@REDACTED Tue Mar 1 19:01:16 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Tue, 1 Mar 2011 15:01:16 -0300 Subject: [erlang-questions] catch In-Reply-To: References: Message-ID: Just use the returned value. Try this: main([String]) -> case length(String) of 0 -> io:format("~nDid you forget the number?~n~n"); _ -> case catch(list_to_integer(String)) of {'EXIT', {badarg,_}} -> io:format("~n~p~n~n",["Arg not Int"]); *I ->* Seq=lists:seq(I+1, I+20), lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) end end; main(_) -> io:format("~nDid you forget the number?~n~n"). On Tue, Mar 1, 2011 at 2:46 PM, Wes James wrote: > I have this escript to create a list of output: > > main([String]) -> > case length(String) of > 0 -> io:format("~nDid you forget the number?~n~n"); > _ -> > case catch(list_to_integer(String)) of > {'EXIT', {badarg,_}} -> > io:format("~n~p~n~n",["Arg not > Int"]); > _ -> > I=list_to_integer(String), > Seq=lists:seq(I+1, I+20), > lists:foreach(fun(H) -> > io:format("#./runcscript ~p ~n",[H]) end, Seq) > end > end; > > main(_) -> > io:format("~nDid you forget the number?~n~n"). > > > I first tried: > > case catch(I=list_to_integer(String)) of > > But I think there was an error that I was unsafe. > > Is there a way to do a variable assignment to retain a good value so > you don't have to do the catch then redo it when it is valid? I.e., > list_to_ineger will be run twice if it is a valid integer. Is there a > way to do this once? > > thx, > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- *Fernando Benavides * From lukas.larsson@REDACTED Tue Mar 1 19:01:23 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Tue, 1 Mar 2011 18:01:23 +0000 (GMT) Subject: [erlang-questions] catch In-Reply-To: Message-ID: <1471064106.8451299002483388.JavaMail.root@zimbra> You can match it out in the case, or maybe have a look at try ... catch ... end in the documentation. case catch(list_to_integer(String)) of {'EXIT', {badarg,_}} -> io:format("~n~p~n~n",["Arg not Int"]); I -> do_stuff(I) end Lukas ----- Original Message ----- From: "Wes James" To: erlang-questions@REDACTED Sent: Tuesday, 1 March, 2011 18:46:34 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] catch I have this escript to create a list of output: main([String]) -> case length(String) of 0 -> io:format("~nDid you forget the number?~n~n"); _ -> case catch(list_to_integer(String)) of {'EXIT', {badarg,_}} -> io:format("~n~p~n~n",["Arg not Int"]); _ -> I=list_to_integer(String), Seq=lists:seq(I+1, I+20), lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) end end; main(_) -> io:format("~nDid you forget the number?~n~n"). I first tried: case catch(I=list_to_integer(String)) of But I think there was an error that I was unsafe. Is there a way to do a variable assignment to retain a good value so you don't have to do the catch then redo it when it is valid? I.e., list_to_ineger will be run twice if it is a valid integer. Is there a way to do this once? thx, -wes ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From comptekki@REDACTED Tue Mar 1 19:15:29 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 1 Mar 2011 11:15:29 -0700 Subject: [erlang-questions] catch In-Reply-To: References: Message-ID: On Tue, Mar 1, 2011 at 11:01 AM, Fernando Benavides wrote: > Just use the returned value. Try this: > main([String]) -> > ?? case length(String) of > ?? ? ? 0 -> io:format("~nDid you forget the number?~n~n"); > ?? ? ? _ -> > ?? ? ? ? ? case catch(list_to_integer(String)) of > ?? ? ? ? ? ? ? {'EXIT', {badarg,_}} -> > ?? ? ? ? ? ? ? ? ? io:format("~n~p~n~n",["Arg not Int"]); > ?? ? ? ? ? ? ? I -> > ?? ? ? ? ? ? ? ? ??Seq=lists:seq(I+1, I+20), > ?? ? ? ? ? ? ? ? ? lists:foreach(fun(H) -> io:format("#./runcscript ~p > ~n",[H]) end, Seq) > ?? ? ? ? ? end > ?? end; > main(_) -> > ?? io:format("~nDid you forget the number?~n~n"). On Tue, Mar 1, 2011 at 11:01 AM, Fernando Benavides wrote: > Just use the returned value. Try this: > main([String]) -> > case length(String) of > 0 -> io:format("~nDid you forget the number?~n~n"); > _ -> > case catch(list_to_integer(String)) of > {'EXIT', {badarg,_}} -> > io:format("~n~p~n~n",["Arg not Int"]); > I -> > Seq=lists:seq(I+1, I+20), > lists:foreach(fun(H) -> io:format("#./runcscript ~p > ~n",[H]) end, Seq) > end > end; > main(_) -> > io:format("~nDid you forget the number?~n~n"). Thx Frenando - that makes sense :) -wes From comptekki@REDACTED Tue Mar 1 19:24:46 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 1 Mar 2011 11:24:46 -0700 Subject: [erlang-questions] catch In-Reply-To: <1471064106.8451299002483388.JavaMail.root@zimbra> References: <1471064106.8451299002483388.JavaMail.root@zimbra> Message-ID: On Tue, Mar 1, 2011 at 11:01 AM, Lukas Larsson wrote: > You can match it out in the case, or maybe have a look at try ... catch ... end in the documentation. > > case catch(list_to_integer(String)) of > ? {'EXIT', {badarg,_}} -> > ? ? ? ?io:format("~n~p~n~n",["Arg not Int"]); > ? ?I -> > ? ? ? ?do_stuff(I) > end > > Lukas thx, Lukas - that is similar to Fernando's answer and is just what I needed. -wes From bob@REDACTED Tue Mar 1 20:46:05 2011 From: bob@REDACTED (Bob Cowdery) Date: Tue, 01 Mar 2011 19:46:05 +0000 Subject: inet:ifget("eth0", [broadaddr]) Message-ID: <4D6D4CFD.2050307@bobcowdery.plus.com> I'm trying to get hold of the broadcast address on a Windows box but inet:ifget() just returns einval. Anyone know of a way to do this. Thanks Bob From juanjo@REDACTED Tue Mar 1 20:57:00 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 1 Mar 2011 16:57:00 -0300 Subject: [erlang-questions] inet:ifget("eth0", [broadaddr]) In-Reply-To: <4D6D4CFD.2050307@bobcowdery.plus.com> References: <4D6D4CFD.2050307@bobcowdery.plus.com> Message-ID: I doubt that the Ethernet interface is called 'eth0' on Windows. What does inet:getiflist() return on your system? Juanjo On Tue, Mar 1, 2011 at 4:46 PM, Bob Cowdery wrote: > I'm trying to get hold of the broadcast address on a Windows box but > inet:ifget() just returns einval. Anyone know of a way to do this. > > Thanks > Bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From qwertymaniac@REDACTED Tue Mar 1 21:14:58 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Wed, 2 Mar 2011 01:44:58 +0530 Subject: [erlang-questions] inet:ifget("eth0", [broadaddr]) In-Reply-To: <4D6D4CFD.2050307@bobcowdery.plus.com> References: <4D6D4CFD.2050307@bobcowdery.plus.com> Message-ID: Hello, On Wed, Mar 2, 2011 at 1:16 AM, Bob Cowdery wrote: > I'm trying to get hold of the broadcast address on a Windows box but > inet:ifget() just returns einval. Anyone know of a way to do this. While unrelated to the Erlang question here, I would like you to be aware of the ongoing change in the naming scheme for network interfaces. See https://fedoraproject.org/wiki/Features/ConsistentNetworkDeviceNaming for an example, which would invalidate all scripts/code written for finding an "ethX". -- Harsh J www.harshj.com From dmitriy.kargapolov@REDACTED Tue Mar 1 22:29:54 2011 From: dmitriy.kargapolov@REDACTED (Dmitriy Kargapolov) Date: Tue, 1 Mar 2011 16:29:54 -0500 Subject: process memory and performance In-Reply-To: References: Message-ID: On Fri, Feb 11, 2011 at 2:46 PM, Dmitriy Kargapolov < dmitriy.kargapolov@REDACTED> wrote: > Hi all, > I have some process (gen_sever) performing multiple mnesia queries, so > complex lookup takes about 840 mks (pretty old h/w, non-smp, R11B-5). When > internal in-memory storage added to this process, so its memory increased up > to ~ 60MB, same lookup slow down to 2200 mks. Note that in this test lookup > did not use big memory. This big memory organized as complex recursive data > structure. > Why memory allocated by process indirectly affects its performance? Is it > related to how GC works? > Thanks. > Revisiting this issue I found that previous developers of the project did set 'fullsweep_after' gc parameter to 15, probably trying to reduce memory consumption... Setting it back to default value (65535) I have "solved" the problem. From bob@REDACTED Tue Mar 1 22:47:45 2011 From: bob@REDACTED (Bob Cowdery) Date: Tue, 01 Mar 2011 21:47:45 +0000 Subject: [erlang-questions] inet:ifget("eth0", [broadaddr]) In-Reply-To: <4D6D4CFD.2050307@bobcowdery.plus.com> References: <4D6D4CFD.2050307@bobcowdery.plus.com> Message-ID: <4D6D6981.1060306@bobcowdery.plus.com> I've found another function inet:getifaddrs/0 which returns a host of information: {ok,[{"\\DEVICE\\TCPIP_{821E4055-EB4F-4E8A-B7F2-30FF9F055679}", [{flags,[up,broadcast,running,multicast]}, {addr,{65152,0,0,0,13375,30402,12581,1246}}, {netmask,{65535,65535,65535,65535,0,0,0,0}}, {addr,{192,168,1,65}}, {netmask,{255,255,255,0}}, {broadaddr,{192,168,1,255}}, {hwaddr,[0,25,209,164,163,180]}]}, {"\\DEVICE\\TCPIP_{0D3F6CDF-8C9C-441B-AE00-70C76036CE1B}", [{flags,[up,broadcast,running,multicast]}, {addr,{65152,0,0,0,6586,36203,15466,30550}}, {netmask,{65535,65535,65535,65535,0,0,0,0}}, {addr,{192,168,56,1}}, {netmask,{255,255,255,0}}, {broadaddr,{192,168,56,255}}, {hwaddr,[8,0,39,0,24,23]}]}, {"\\DEVICE\\TCPIP_{E29AC6C2-7037-11DE-816D-806E6F6E6963}", [{flags,[up,loopback,running]}, {addr,{0,0,0,0,0,0,0,1}}, {netmask,{65535,65535,65535,65535,65535,65535,65535,65535}}, {addr,{127,0,0,1}}, {netmask,{255,0,0,0}}]}, {"\\DEVICE\\TCPIP_{D492CD99-E0DE-41CE-9872-FE16A171110D}", [{flags,[up]}, {addr,{65152,0,0,0,0,24318,49320,321}}, {netmask,{65535,65535,65535,65535,65535,65535,65535,65535}}, {hwaddr,[0,0,0,0,0,0,0,224]}]}, {"\\DEVICE\\TCPIP_{CDBD6FCC-4871-4D01-8B5D-E1EC67B4D776}", [{flags,[up,running]}, {addr,{8193,0,24309,31229,6352,13625,11104,54066}}, {netmask,{0,0,0,0,0,0,0,0}}, {addr,{65152,0,0,0,6352,13625,11104,54066}}, {netmask,{0,0,0,0,0,0,0,0}}, {hwaddr,[0,0,0,0,0,0,0,224]}]}, {"\\DEVICE\\TCPIP_{24F809A3-F9E8-4A77-83AE-3D00133FDC71}", [{flags,[up]},{hwaddr,[0,0,0,0,0,0,0,224]}]}]} Plugging those somewhat ridiculous names into inet:ifget/0 still gives einval but at least I can extract the broadaddr from the above structure. Bob On 01/03/2011 19:46, Bob Cowdery wrote: > I'm trying to get hold of the broadcast address on a Windows box but > inet:ifget() just returns einval. Anyone know of a way to do this. > > Thanks > Bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bob@REDACTED Tue Mar 1 21:50:21 2011 From: bob@REDACTED (Bob Cowdery) Date: Tue, 01 Mar 2011 20:50:21 +0000 Subject: [erlang-questions] inet:ifget("eth0", [broadaddr]) In-Reply-To: References: <4D6D4CFD.2050307@bobcowdery.plus.com> Message-ID: <4D6D5C0D.5050907@bobcowdery.plus.com> I doubt it is too. But I couldn't determine if it actually had a name or where to find it. inet:getiflist() just returns {ok,["192.168.1.65","192.168.56.1","127.0.0.1"]} which doesn't really help. Bob On 01/03/2011 19:57, Juan Jose Comellas wrote: > I doubt that the Ethernet interface is called 'eth0' on Windows. What > does inet:getiflist() return on your system? > > Juanjo > > > On Tue, Mar 1, 2011 at 4:46 PM, Bob Cowdery wrote: > >> I'm trying to get hold of the broadcast address on a Windows box but >> inet:ifget() just returns einval. Anyone know of a way to do this. >> >> Thanks >> Bob >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From watson.timothy@REDACTED Tue Mar 1 23:19:17 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 1 Mar 2011 22:19:17 +0000 Subject: [erlang-questions] is anyone else getting these one-question mails? In-Reply-To: References: <20110301163305.GA3473@corelatus.se> Message-ID: On 1 March 2011 16:39, Gleb Peregud wrote: > On Tue, Mar 1, 2011 at 17:33, Matthias Lang wrote: >> Is anyone else getting these? > No > >> It feels a bit like a bot. > Yeah, it feels like bot a bit This had me laughing for longer than I care to admit! From robert.virding@REDACTED Tue Mar 1 23:29:59 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 1 Mar 2011 22:29:59 +0000 (GMT) Subject: [erlang-questions] is anyone else getting these one-question mails? In-Reply-To: Message-ID: <630370050.8681299018599269.JavaMail.root@zimbra> ----- "Tim Watson" wrote: > On 1 March 2011 16:39, Gleb Peregud wrote: > > On Tue, Mar 1, 2011 at 17:33, Matthias Lang > wrote: > >> Is anyone else getting these? > > No > > > >> It feels a bit like a bot. > > Yeah, it feels like bot a bit > > This had me laughing for longer than I care to admit! Why did it have you laughing for longer than you care to admit? From michaelmackus@REDACTED Tue Mar 1 23:46:12 2011 From: michaelmackus@REDACTED (Michael Mackus) Date: Tue, 1 Mar 2011 14:46:12 -0800 Subject: Erlang Win32com Message-ID: Is there a way to access win32com using erlang? For instance, in python, there's the "win32com" module. If you don't understand, I need a way to access functions in a windows DLL from erlang. This is for a bot project I'm doing if you are curious, seems like it'll get my feet wet with using Erlang. From kvs@REDACTED Tue Mar 1 23:51:27 2011 From: kvs@REDACTED (Konstantin Sorokin) Date: Wed, 2 Mar 2011 01:51:27 +0300 Subject: Why do people keep rebar binary in repos ? Message-ID: Hello! I noticed that a lot of people keep 'rebar' in their public repositories. I wonder what may be the reason behind that ? I am coming from the C++ world and for me it seems very strange like, say, keeping 'make' binary along with the C++ sources. -- Konstantin Sorokin From jack@REDACTED Wed Mar 2 00:06:58 2011 From: jack@REDACTED (Jack Moffitt) Date: Tue, 1 Mar 2011 16:06:58 -0700 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: Message-ID: > I noticed that a lot of people keep 'rebar' in their public repositories. I > wonder what may be the reason behind that ? I am coming from the C++ world > and for me it seems very strange like, say, keeping 'make' binary along with > the C++ sources. make is ubiquitous; rebar is not. It's a far better user experience to let users type "rebar" than to make them download and install it as well. This is similar to how autotools projects include prebuilt configure scripts as opposed to making people download and run autoconf, automake, and libtool themselves. jack. From dizzyd@REDACTED Wed Mar 2 00:08:23 2011 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 1 Mar 2011 16:08:23 -0700 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: Message-ID: On Tue, Mar 1, 2011 at 3:51 PM, Konstantin Sorokin wrote: > Hello! > > I noticed that a lot of people keep 'rebar' in their public repositories. I > wonder what may be the reason behind that ? I am coming from the C++ world > and for me it seems very strange like, say, keeping 'make' binary along with > the C++ sources. Generally, this was something I (and the rebar team, generally) encouraged for reasons of adoption. Requiring someone to install a new (unknown) tool like rebar just to compile code is rarely a popular way to get a project bootstrapped. :) However, with the current adoption level, this is something I expect will start changing. Yes, it is a little odd but it sure was handy. :) D. From ok@REDACTED Wed Mar 2 00:37:47 2011 From: ok@REDACTED (ok@REDACTED) Date: Wed, 2 Mar 2011 12:37:47 +1300 Subject: [erlang-questions] catch In-Reply-To: References: Message-ID: <4f055745c8c2f3be9d31fd8a02ce56ba.squirrel@chasm.otago.ac.nz> > I have this escript to create a list of output: > > main([String]) -> > case length(String) of > 0 -> io:format("~nDid you forget the number?~n~n"); > _ -> ... Just as a matter of curiosity, why not write main([String]) -> case String of [] -> ... ; _ -> ... end. or even main([""]) -> ...; main([String]) -> ... Also, why are you calling list_to_integer/1 twice? case catch list_to_integer(String) of {'EXIT',{badarg,_}} -> ... ; I -> ... end You did *try* typing 1> catch list_to_integer("fred"). 2> catch list_to_integer("100"). didn't you? From norton@REDACTED Wed Mar 2 00:40:59 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Wed, 2 Mar 2011 08:40:59 +0900 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: Message-ID: Konstantin - It is (also) handy to have the rebar binary available if customizations to rebar itself are needed for a particular project. On 2011/03/02, at 7:51, Konstantin Sorokin wrote: > Hello! > > I noticed that a lot of people keep 'rebar' in their public repositories. I > wonder what may be the reason behind that ? I am coming from the C++ world > and for me it seems very strange like, say, keeping 'make' binary along with > the C++ sources. > > -- > Konstantin Sorokin Joseph Wayne Norton norton@REDACTED From rzezeski@REDACTED Wed Mar 2 01:01:22 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 1 Mar 2011 19:01:22 -0500 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: Message-ID: I'll also add rebar is still a bit of a moving target and seems to get upgraded often. Having it right there makes sure you have the right version. -Ryan [Sent from my iPhone] On Mar 1, 2011, at 6:40 PM, Joseph Wayne Norton wrote: > > Konstantin - > > It is (also) handy to have the rebar binary available if customizations to rebar itself are needed for a particular project. > > On 2011/03/02, at 7:51, Konstantin Sorokin wrote: > >> Hello! >> >> I noticed that a lot of people keep 'rebar' in their public repositories. I >> wonder what may be the reason behind that ? I am coming from the C++ world >> and for me it seems very strange like, say, keeping 'make' binary along with >> the C++ sources. >> >> -- >> Konstantin Sorokin > > Joseph Wayne Norton > norton@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ok@REDACTED Wed Mar 2 04:08:49 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 Mar 2011 16:08:49 +1300 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: <511794668.258121294521466549.JavaMail.root@zimbra> <179103885.258141294521619929.JavaMail.root@zimbra> Message-ID: On 22/01/2011, at 5:59 AM, Michael Turner wrote: > win. But when you're maintaining a large, sprawling, legacy system, you're > not doing it in that system's (main) language because the language is > popular -- I've even had to do maintenance in Fortran, a language that, as > far as I can tell, was unpopular almost the day it was born. No, you're > doing it because to do otherwise is either impossible or so awkward at the > interfaces as to not be worth it, and/or because you're asking other > programmers to learn and like the new language you choose even when it is > possible to incrementally change the code base in the new language, and/or > because to start rewriting the system from scratch is nowhere near an > option, financially, for the profit-making (you hope) entity you work for. There is more than one kind of maintenance. If you are fixing mistakes, you probably need to fix them in the original language. (Although it must be said that modern Fortran is quite a pleasant modular language which has a lot to recommend it.) If you are adding new features, it might be an excellent idea to use some other language. For an Erlang example, the AXE-10 switch was programmed in about a million lines of EriPascal. Once you get to that sort of size in a language like that, every time you change one thing, you probably break something else. So one bright person (whose name is on the tip of my tongue but won't come off) came up with the idea of adding once and for all a control stub and adding new features by programming them in Erlang and sending control messages through the stub. The narrow coupling greatly reduced the potential for introducing new bugs, so this was _better_ than hacking in the original code base. From anthonym@REDACTED Wed Mar 2 08:01:47 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 1 Mar 2011 23:01:47 -0800 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: Message-ID: <20110302070147.GE30998@alumni.caltech.edu> One thing which might help adoption by systems packagers (ie, debian, ubuntu, redhat, epel) is versioned releases. The only tag right now on github is 'RELEASE-1' from 2009. And you can download 'rebar' then ask it for it's version but as I only see one rebar that either means, it doesn't change, so I should build from source, or it changes frequently but is always named the same thing (at which point I have no way to get back to an old version if I need to). For my current RPM I use at work, I checked out on a particular day, and used the day as my version number. I also noted the id of the last commit and put that into my scripts which recreate the RPM, so in case I ever need to rebuild it, I know where I started. Better would be if some form of semantic versioning were adopted and things tagged, and the version as part of the download. That way I could see downloads like 'rebar-1.0.2', or 'rebar-1.5.0'. Once it's downloaded I can easily symlink 'rebar' to the appropriate version (or have multiple versions installed if I needed to). Anyway, I realize software is never "done", but it is useful for the different packagers if lines are drawn in the sand every so often. -Anthony On Tue, Mar 01, 2011 at 04:08:23PM -0700, Dave Smith wrote: > On Tue, Mar 1, 2011 at 3:51 PM, Konstantin Sorokin wrote: > > Hello! > > > > I noticed that a lot of people keep 'rebar' in their public repositories. I > > wonder what may be the reason behind that ? I am coming from the C++ world > > and for me it seems very strange like, say, keeping 'make' binary along with > > the C++ sources. > > Generally, this was something I (and the rebar team, generally) > encouraged for reasons of adoption. Requiring someone to install a new > (unknown) tool like rebar just to compile code is rarely a popular way > to get a project bootstrapped. :) However, with the current adoption > level, this is something I expect will start changing. > > Yes, it is a little odd but it sure was handy. :) > > D. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From sedrik@REDACTED Wed Mar 2 08:50:08 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Wed, 2 Mar 2011 08:50:08 +0100 Subject: [erlang-questions] is anyone else getting these one-question mails? In-Reply-To: <630370050.8681299018599269.JavaMail.root@zimbra> References: <630370050.8681299018599269.JavaMail.root@zimbra> Message-ID: Could this be an attempt to bypass spam filters by establishing a "relationship" with the potential spam victim?Or why would a bot send messages like these around?Or your mail has been published on some unknown message board in a country where people do not use spaces after punctuation. Note the added fun of me not having spaces after my punctuations :) On Tue, Mar 1, 2011 at 11:29 PM, Robert Virding < robert.virding@REDACTED> wrote: > ----- "Tim Watson" wrote: > > > On 1 March 2011 16:39, Gleb Peregud wrote: > > > On Tue, Mar 1, 2011 at 17:33, Matthias Lang > > wrote: > > >> Is anyone else getting these? > > > No > > > > > >> It feels a bit like a bot. > > > Yeah, it feels like bot a bit > > > > This had me laughing for longer than I care to admit! > > Why did it have you laughing for longer than you care to admit? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From raimo+erlang-questions@REDACTED Wed Mar 2 09:17:12 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 2 Mar 2011 09:17:12 +0100 Subject: [erlang-questions] inet:ifget("eth0", [broadaddr]) In-Reply-To: <4D6D6981.1060306@bobcowdery.plus.com> References: <4D6D4CFD.2050307@bobcowdery.plus.com> <4D6D6981.1060306@bobcowdery.plus.com> Message-ID: <20110302081712.GA5127@erix.ericsson.se> On Tue, Mar 01, 2011 at 09:47:45PM +0000, Bob Cowdery wrote: > I've found another function inet:getifaddrs/0 which returns a host of > information: > > {ok,[{"\\DEVICE\\TCPIP_{821E4055-EB4F-4E8A-B7F2-30FF9F055679}", > [{flags,[up,broadcast,running,multicast]}, > {addr,{65152,0,0,0,13375,30402,12581,1246}}, > {netmask,{65535,65535,65535,65535,0,0,0,0}}, > {addr,{192,168,1,65}}, > {netmask,{255,255,255,0}}, > {broadaddr,{192,168,1,255}}, > {hwaddr,[0,25,209,164,163,180]}]}, > {"\\DEVICE\\TCPIP_{0D3F6CDF-8C9C-441B-AE00-70C76036CE1B}", > [{flags,[up,broadcast,running,multicast]}, > {addr,{65152,0,0,0,6586,36203,15466,30550}}, > {netmask,{65535,65535,65535,65535,0,0,0,0}}, > {addr,{192,168,56,1}}, > {netmask,{255,255,255,0}}, > {broadaddr,{192,168,56,255}}, > {hwaddr,[8,0,39,0,24,23]}]}, > {"\\DEVICE\\TCPIP_{E29AC6C2-7037-11DE-816D-806E6F6E6963}", > [{flags,[up,loopback,running]}, > {addr,{0,0,0,0,0,0,0,1}}, > {netmask,{65535,65535,65535,65535,65535,65535,65535,65535}}, > {addr,{127,0,0,1}}, > {netmask,{255,0,0,0}}]}, > {"\\DEVICE\\TCPIP_{D492CD99-E0DE-41CE-9872-FE16A171110D}", > [{flags,[up]}, > {addr,{65152,0,0,0,0,24318,49320,321}}, > {netmask,{65535,65535,65535,65535,65535,65535,65535,65535}}, > {hwaddr,[0,0,0,0,0,0,0,224]}]}, > {"\\DEVICE\\TCPIP_{CDBD6FCC-4871-4D01-8B5D-E1EC67B4D776}", > [{flags,[up,running]}, > {addr,{8193,0,24309,31229,6352,13625,11104,54066}}, > {netmask,{0,0,0,0,0,0,0,0}}, > {addr,{65152,0,0,0,6352,13625,11104,54066}}, > {netmask,{0,0,0,0,0,0,0,0}}, > {hwaddr,[0,0,0,0,0,0,0,224]}]}, > {"\\DEVICE\\TCPIP_{24F809A3-F9E8-4A77-83AE-3D00133FDC71}", > [{flags,[up]},{hwaddr,[0,0,0,0,0,0,0,224]}]}]} > > Plugging those somewhat ridiculous names into inet:ifget/0 still gives > einval but at least I can extract the broadaddr from the above structure. See the documentation for inet:getifaddrs/0 and notice that it is actually documented, while inet:ifget/0,1 is not. The latter and its associate inet:getiflist/0 are legacy functions that front Unixish IOCtls. These have been abandoned by the Unixish communities in favour of getifaddrs()-like interfaces where you get all info in one call and they are the way to get IPv6 info. On Windows, OS version specific interfaces are used to mimic getifaddrs(), so you get a bit different info on e.g Windows XP; you seem to be running Vista or 7 with all that IPv6 info. To make Erlang's legacy function work better was to complicated and would mean having to call a getifaddrs()-like function and filter out the item asked for for every call, so they are left as they are and used internally by inet:getifaddrs/0 as a fallback. inet:getifaddrs/0 is the only reliable and reasonably portable function to use for network interface information retrieval, and not very hard to parse. > > Bob > > On 01/03/2011 19:46, Bob Cowdery wrote: > > I'm trying to get hold of the broadcast address on a Windows box but > > inet:ifget() just returns einval. Anyone know of a way to do this. > > > > Thanks > > Bob > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From attila.r.nohl@REDACTED Wed Mar 2 10:07:25 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 2 Mar 2011 10:07:25 +0100 Subject: [erlang-questions] catch In-Reply-To: References: Message-ID: 2011/3/1, Wes James : > I have this escript to create a list of output: > > main([String]) -> > case length(String) of > 0 -> io:format("~nDid you forget the number?~n~n"); > _ -> > case catch(list_to_integer(String)) of > {'EXIT', {badarg,_}} -> > io:format("~n~p~n~n",["Arg not Int"]); > _ -> > I=list_to_integer(String), > Seq=lists:seq(I+1, I+20), > lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) > end > end; > > main(_) -> > io:format("~nDid you forget the number?~n~n"). Haven't tested (or even compiled), but a lot clearer: main([""]) -> io:format("~nDid you forget the number?~n~n"); main([String]) -> try I=list_to_integer(String), Seq=lists:seq(I+1, I+20), lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) catch error:{badarg,_} -> io:format("~n~p~n~n",["Arg not Int"]) end; main(_) -> io:format("~nDid you forget the number?~n~n"). From michael.eugene.turner@REDACTED Wed Mar 2 12:22:44 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 2 Mar 2011 20:22:44 +0900 Subject: Rubyists' #1 "wanna-learn" language? Maybe it's Erlang Message-ID: And if so, perhaps by a wide margin. Interesting small-sample poll: http://weblog.jamisbuck.org/2011/2/22/programming-language-survey-results -michael turner From iostres@REDACTED Wed Mar 2 12:49:38 2011 From: iostres@REDACTED (Ivan Ostres) Date: Wed, 2 Mar 2011 12:49:38 +0100 Subject: Erlang SIGTRAN implementation In-Reply-To: References: , Message-ID: Hi All! Is there such thing as open source SIGTRAN implementation in Erlang? Regards, Ivan Ostres From gleber.p@REDACTED Wed Mar 2 13:43:05 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 2 Mar 2011 13:43:05 +0100 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: <20110302070147.GE30998@alumni.caltech.edu> References: <20110302070147.GE30998@alumni.caltech.edu> Message-ID: > Better would be if some form of semantic versioning were adopted and > things tagged, and the version as part of the download. Take a look at http://semver.org/ From orionqwest@REDACTED Wed Mar 2 15:16:17 2011 From: orionqwest@REDACTED (Nav) Date: Wed, 2 Mar 2011 16:16:17 +0200 Subject: Disallowing gen_server:handle_cast() for a while? Message-ID: Hi, Is there a way to stop handle_cast(..) being called (i.e. stop gen_server from reading data from queue) for a while? I am trying to implement a logic, while processing, gen_server should not accept any messages and let messages stay in its queue. (Other processes can still send messages using gen_server:cast()) Does gen_server maintain separate message queue for each of its handle_cast? While processing handle_cast(message1, ..), could it possibly process handle_cast(message2,...) in parallel, if there is such message in queue? (What would in-built concurrency in gen_server mean exactly?) cheers, -Nav From matthew@REDACTED Wed Mar 2 15:20:28 2011 From: matthew@REDACTED (Matthew Sackman) Date: Wed, 2 Mar 2011 14:20:28 +0000 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: Message-ID: <20110302142027.GA31532@rabbitmq.com> On Wed, Mar 02, 2011 at 04:16:17PM +0200, Nav wrote: > Is there a way to stop handle_cast(..) being called (i.e. stop > gen_server from reading data from queue) for a while? A selective receive? A timer:sleep? > Does gen_server maintain separate message queue for each of its handle_cast? No, each process has one mailbox. That is true regardless of the use of gen_server. Matthew From matthew@REDACTED Wed Mar 2 15:34:42 2011 From: matthew@REDACTED (Matthew Sackman) Date: Wed, 2 Mar 2011 14:34:42 +0000 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: <20110302142027.GA31532@rabbitmq.com> Message-ID: <20110302143442.GB31532@rabbitmq.com> On Wed, Mar 02, 2011 at 04:27:41PM +0200, Nav wrote: > Does gen_sever ensure that while I am executing > > handle_cast(message1,...) -> {no spawn ..etc} > > the next message stays in the mailbox? Yes. Each process is a single process/thread. It cannot do multiple things at the same time. > timer:sleep is not an option since my logic would be inside gen_server itself. It's rare to have good reason to alter gen_server's code yourself, though that might not be what you're suggesting. > My main requirement > > handle_cast(message1,...) -> > > %% force gen_server to not read any message from queue > > %% some actions > > end. > %% now gen_server can read message as before. That's exactly what it does. Matthew From max.lapshin@REDACTED Wed Mar 2 15:59:25 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 2 Mar 2011 17:59:25 +0300 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: <20110302143442.GB31532@rabbitmq.com> References: <20110302142027.GA31532@rabbitmq.com> <20110302143442.GB31532@rabbitmq.com> Message-ID: I think you need something like sys:suspend (or how is it called?) From allen.kim@REDACTED Wed Mar 2 16:02:52 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 2 Mar 2011 09:02:52 -0600 Subject: Have been looking for mock and stub Message-ID: I have been looking for mock and stub for eunit testing, but I couldn't find a simple one. So, I build one by borrowing few codes from open source, and used it in our company for a year and I thought it was good enough. Here is the code that I use and share, https://github.com/epicadvertising/mock_stub And the following is the example of usage; Eshell V5.7.3 (abort with ^G) > {ok, _Pid} = mock:start(my_module). > mock:expect(my_module, foo, [], mock_foo0). > mock:expect(my_module, 'foo/1', mock_other_foo1). > my_module:foo(). mock_foo0 > my_module:foo(1). mock_other_foo1 > mock:stop(my_module). > my_module:foo(). foo0 > gen_server:start({local, my_reg_gen_server}, my_gen_server, [], []). > gen_server:call(my_reg_gen_server, foo). call_foo > mock:start(my_gen_server). > mock:expect(my_gen_server, 'handle_call/3', {reply, mock_call_foo, []}). > gen_server:call(my_reg_gen_server, foo). mock_call_foo > gen_event:start_link({local,my_gen_event_manager}). > gen_event:add_handler(my_gen_event_manager, my_gen_event, []). > gen_event:call(my_gen_event_manager, my_gen_event, foo). call_foo > mock:start(my_gen_event). > mock:expect(my_gen_event, 'handle_call/2', {ok, mock_call_foo, []}). > gen_event:call(my_gen_event_manager, my_gen_event, foo). mock_call_foo I hope this is useful for somebody who is looking for simple mock and stub, and if not please lt me know. Allen From michal.ptaszek@REDACTED Wed Mar 2 16:20:31 2011 From: michal.ptaszek@REDACTED (Michal Ptaszek) Date: Wed, 2 Mar 2011 16:20:31 +0100 Subject: [erlang-questions] Have been looking for mock and stub In-Reply-To: References: Message-ID: <60382950-D804-46D8-8CCE-1B79106A2196@erlang-solutions.com> By any chance, have you tried meck? https://github.com/esl/meck On Mar 2, 2011, at 4:02 PM, Allen Kim wrote: > I have been looking for mock and stub for eunit testing, but I couldn't find a simple one. > So, I build one by borrowing few codes from open source, and used it in our company for a year and I thought it was good enough. > > Here is the code that I use and share, https://github.com/epicadvertising/mock_stub > And the following is the example of usage; > > > Eshell V5.7.3 (abort with ^G) > >> {ok, _Pid} = mock:start(my_module). >> mock:expect(my_module, foo, [], mock_foo0). >> mock:expect(my_module, 'foo/1', mock_other_foo1). >> my_module:foo(). > mock_foo0 >> my_module:foo(1). > mock_other_foo1 >> mock:stop(my_module). >> my_module:foo(). > foo0 > >> gen_server:start({local, my_reg_gen_server}, my_gen_server, [], []). >> gen_server:call(my_reg_gen_server, foo). > call_foo >> mock:start(my_gen_server). >> mock:expect(my_gen_server, 'handle_call/3', {reply, mock_call_foo, []}). >> gen_server:call(my_reg_gen_server, foo). > mock_call_foo > >> gen_event:start_link({local,my_gen_event_manager}). >> gen_event:add_handler(my_gen_event_manager, my_gen_event, []). >> gen_event:call(my_gen_event_manager, my_gen_event, foo). > call_foo >> mock:start(my_gen_event). >> mock:expect(my_gen_event, 'handle_call/2', {ok, mock_call_foo, []}). >> gen_event:call(my_gen_event_manager, my_gen_event, foo). > mock_call_foo > > > I hope this is useful for somebody who is looking for simple mock and stub, and if not please lt me know. > > Allen From allen.kim@REDACTED Wed Mar 2 16:32:13 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 2 Mar 2011 09:32:13 -0600 Subject: [erlang-questions] Have been looking for mock and stub In-Reply-To: <60382950-D804-46D8-8CCE-1B79106A2196@erlang-solutions.com> Message-ID: Thanks for the link. no, I haven't I wanted to achieve three things from my code; mock, stub, and OTP behaviours (gen_server, gen_event, gen_fsm). I will test your suggestoin and check if it also covers all of them. Allen From: Michal Ptaszek > Date: Wed, 2 Mar 2011 09:20:31 -0600 To: Allen Kim > Cc: "erlang-questions@REDACTED" > Subject: Re: [erlang-questions] Have been looking for mock and stub By any chance, have you tried meck? https://github.com/esl/meck On Mar 2, 2011, at 4:02 PM, Allen Kim wrote: I have been looking for mock and stub for eunit testing, but I couldn't find a simple one. So, I build one by borrowing few codes from open source, and used it in our company for a year and I thought it was good enough. Here is the code that I use and share, https://github.com/epicadvertising/mock_stub And the following is the example of usage; Eshell V5.7.3 (abort with ^G) {ok, _Pid} = mock:start(my_module). mock:expect(my_module, foo, [], mock_foo0). mock:expect(my_module, 'foo/1', mock_other_foo1). my_module:foo(). mock_foo0 my_module:foo(1). mock_other_foo1 mock:stop(my_module). my_module:foo(). foo0 gen_server:start({local, my_reg_gen_server}, my_gen_server, [], []). gen_server:call(my_reg_gen_server, foo). call_foo mock:start(my_gen_server). mock:expect(my_gen_server, 'handle_call/3', {reply, mock_call_foo, []}). gen_server:call(my_reg_gen_server, foo). mock_call_foo gen_event:start_link({local,my_gen_event_manager}). gen_event:add_handler(my_gen_event_manager, my_gen_event, []). gen_event:call(my_gen_event_manager, my_gen_event, foo). call_foo mock:start(my_gen_event). mock:expect(my_gen_event, 'handle_call/2', {ok, mock_call_foo, []}). gen_event:call(my_gen_event_manager, my_gen_event, foo). mock_call_foo I hope this is useful for somebody who is looking for simple mock and stub, and if not please lt me know. Allen From als@REDACTED Wed Mar 2 18:30:44 2011 From: als@REDACTED (Anthony Shipman) Date: Thu, 3 Mar 2011 04:30:44 +1100 Subject: [erlang-questions] Success Stories of Erlang Development In-Reply-To: <4D6CFBD3.8000708@erlang-solutions.com> References: <810922686.10234471298947933374.JavaMail.root@ucimail4.uci.cu> <4D6CFBD3.8000708@erlang-solutions.com> Message-ID: <201103030430.44279.als@iinet.net.au> The product described at www.symstream.com is mostly Erlang on the server side. -- Anthony Shipman Mamas don't let your babies als@REDACTED grow up to be outsourced. From antoine.koener@REDACTED Wed Mar 2 18:38:35 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Wed, 2 Mar 2011 18:38:35 +0100 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: <20110302142027.GA31532@rabbitmq.com> <20110302143442.GB31532@rabbitmq.com> Message-ID: <134EF07F-FFF8-4E9B-855E-7EC428BC0189@gmail.com> May be you should look for gen_fsm. Jump in a state that stay only for some TIMEOUT, this state will discard any message received... On Mar 2, 2011, at 15:59 , Max Lapshin wrote: > I think you need something like sys:suspend (or how is it called?) > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dhananjay.nene@REDACTED Wed Mar 2 19:02:06 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Wed, 2 Mar 2011 23:32:06 +0530 Subject: (noob-help) Supervision strategies to automatically restart dynamically added children Message-ID: While supervisors are meant to automatically restart failed processes, there is one scenario I am as yet unable figure out which is the idiomatic approach to implement crash recovery under the default OTP scenarios. I have considered a solution, but being a relative newbie, I am not sure if it is idiomatic erlang and if there are better solutions. Question in short : If I have a supervisor which has a number of dynamic children, how do I set up a mechanism where in case of a complete system crash, all the dynamic children restart at the point they were when the system (including the supervisor) crashed. Question in long : ============= Sample Context : A bowling game ------------------------------------------------- Lets say I am writing the software to implement the software necessary to track various games at a bowling alley. I've set up the following processes : a. Lanes : If there are 10 lanes, there are 10 processes, one for each lane. These stay fixed for the entire duration of the program b. Games : A group of players might get together to start a game on a free lane. A new game will get created to track the game through its completion. When the game is over, this process shall terminate c. Players : Each game has a number of players. One process "player_game" is started per player. Sample state of a player game would include current score for the player and if the last two rolls were strike or a spare. For the purpose of brevity, the remainder of this mail only refers to this process and ignores the others Objective : --------------- Assuming this is a single node implementation, if the machine were to crash, upon machine / node restart, all the player_games should be restarted and should be at the point where the player_games were when the machine crashed. Possible supervision strategy : -------------------------------------- 1. Create a simple_one_for_one supervisor player_game_sup which upon starting up for the first time would have no children associated with them. Use supervisor:start_child to start each process 2. The supervisor creates an entry in a database (say mnesia) every time it launches a new process 3. Each player_game updates the entry every time the score gets modified. Upon termination that entry gets deleted 4. Post crash, the supervisor is started again (say after an application restart or via another supervisor) 5. (Here's the difference). By default the supervisor will not restart the dynamically added children (all the player_games). However we modify the init code to inspect the database and launch a player_game for each record it finds. The player_game initialises itself to the current state as in the database and the game(s) can continue where it/they left off. My questions : -------------------- a. Does it make sense to move the responsibility to the supervisor to update the database each time a new player game is started or completed ? b. Is it an idiomatic way to implement crash recovery c. Are there any other perhaps superior ways of implementing this? FWIW : the code I am using to learn erlang is at https://github.com/dnene/bowling . Its not particularly interesting at this stage since it is still under development. Thanks Dhananjay PS: Apologies for posting it to erlang-questions after earlier posting it to erlang programming google group. Those monitoring the latter will receive this question twice. From comptekki@REDACTED Wed Mar 2 20:05:14 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 2 Mar 2011 12:05:14 -0700 Subject: [erlang-questions] catch In-Reply-To: References: Message-ID: On Wed, Mar 2, 2011 at 2:07 AM, Attila Rajmund Nohl wrote: > > Haven't tested (or even compiled), but a lot clearer: > > main([""]) -> > io:format("~nDid you forget the number?~n~n"); > main([String]) -> > try > I=list_to_integer(String), > Seq=lists:seq(I+1, I+20), > lists:foreach(fun(H) -> io:format("#./runcscript ~p ~n",[H]) end, Seq) > catch > error:{badarg,_} -> > io:format("~n~p~n~n",["Arg not Int"]) > end; > main(_) -> > io:format("~nDid you forget the number?~n~n"). > escript script.erl 'a' escript: exception error: bad argument From kenrobinsonster@REDACTED Thu Mar 3 03:17:17 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Thu, 3 Mar 2011 12:17:17 +1000 Subject: Problem Compiling otp_src_R14B01 Message-ID: Hi all, I'm trying to compile otp_src_R14B01 on Ubuntu 10.04 (Lucid). When I run the configure script using a bash shell I get this error: ... checking for odbc in standard locations... -L/usr/lib checking for SQLAllocHandle in -lodbc... no configure: WARNING: "ODBC library - link check failed" configure: creating ./config.status config.status: creating c_src/i686-pc-linux-gnu/Makefile ... ********************************************************************* ********************** APPLICATIONS DISABLED ********************** ********************************************************************* odbc : ODBC library - link check failed ********************************************************************* I have installed libiodbc, odbcinst1debian1, unixodbc and libiodbc2-dev. The libodbc I cannot find. Is there something that I am overlooking? -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From michael.santos@REDACTED Thu Mar 3 03:36:35 2011 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 2 Mar 2011 21:36:35 -0500 Subject: [erlang-questions] Problem Compiling otp_src_R14B01 In-Reply-To: References: Message-ID: <20110303023635.GA14074@ecn.lan> On Thu, Mar 03, 2011 at 12:17:17PM +1000, Ken Robinson wrote: > Hi all, > I'm trying to compile otp_src_R14B01 on Ubuntu 10.04 (Lucid). When I > run the configure script using a bash shell I get this error: > > ... > checking for odbc in standard locations... -L/usr/lib > checking for SQLAllocHandle in -lodbc... no > configure: WARNING: "ODBC library - link check failed" > configure: creating ./config.status > config.status: creating c_src/i686-pc-linux-gnu/Makefile > ... > > ********************************************************************* > ********************** APPLICATIONS DISABLED ********************** > ********************************************************************* > > odbc : ODBC library - link check failed > > ********************************************************************* > > I have installed libiodbc, odbcinst1debian1, unixodbc and > libiodbc2-dev. The libodbc I cannot find. Is there something that I am > overlooking? I use the unixodbc-dev package on Lucid. From jesper.louis.andersen@REDACTED Thu Mar 3 03:43:10 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 3 Mar 2011 03:43:10 +0100 Subject: [erlang-questions] Problem Compiling otp_src_R14B01 In-Reply-To: References: Message-ID: On Thu, Mar 3, 2011 at 03:17, Ken Robinson wrote: > Hi all, > I'm trying to compile otp_src_R14B01 on Ubuntu 10.04 (Lucid). When I > run the configure script using a bash shell I get this error: try "aptitude build-dep erlang" to get the build dependencies satisfied automatically. -- J. From comptekki@REDACTED Thu Mar 3 04:36:04 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 2 Mar 2011 20:36:04 -0700 Subject: [erlang-questions] Book "Concurrent Programming in ERLANG" (2nd Ed.): Source for example programmes? In-Reply-To: References: Message-ID: On Thu, Sep 2, 2010 at 11:17 AM, Boris M?hmer wrote: > I just got my copy of "Concurrent Programming in ERLANG" > (2nd ED; actually "on demand" version by Pearson Education), > and I really love the "2nd Part" of it! > > > Is there anywhere an archive with the example programmes available? > Hmm out of print now :( http://www.pearsonhighered.com/educator/product/Concurrent-Programming-Erlang/9780135083017.page I did find one one amazon.com . A few of the authors are going to be at the San Francisco Erlang Factory. Do you mind signing the book? thx, -wes From kenrobinsonster@REDACTED Thu Mar 3 04:54:01 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Thu, 3 Mar 2011 13:54:01 +1000 Subject: [erlang-questions] Problem Compiling otp_src_R14B01 In-Reply-To: <20110303023635.GA14074@ecn.lan> References: <20110303023635.GA14074@ecn.lan> Message-ID: Thanks I will give that a try next time I build a release. On Thu, Mar 3, 2011 at 12:36 PM, Michael Santos wrote: > On Thu, Mar 03, 2011 at 12:17:17PM +1000, Ken Robinson wrote: >> Hi all, >> I'm trying to compile otp_src_R14B01 on Ubuntu 10.04 (Lucid). When I >> run the configure script using a bash shell I get this error: >> >> ... >> checking for odbc in standard locations... -L/usr/lib >> checking for SQLAllocHandle in -lodbc... no >> configure: WARNING: "ODBC library - link check failed" >> configure: creating ./config.status >> config.status: creating c_src/i686-pc-linux-gnu/Makefile >> ... >> >> ********************************************************************* >> ********************** ?APPLICATIONS DISABLED ?********************** >> ********************************************************************* >> >> odbc ? ? ? ? ? : ODBC library - link check failed >> >> ********************************************************************* >> >> I have installed libiodbc, odbcinst1debian1, unixodbc and >> libiodbc2-dev. The libodbc I cannot find. Is there something that I am >> overlooking? > > I use the unixodbc-dev package on Lucid. > > -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From attila.r.nohl@REDACTED Thu Mar 3 10:57:26 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 3 Mar 2011 10:57:26 +0100 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: Message-ID: 2011/3/2, Nav : > Hi, > > Is there a way to stop handle_cast(..) being called (i.e. stop > gen_server from reading data from queue) for a while? > > I am trying to implement a logic, while processing, gen_server should > not accept any messages and let messages stay in its queue. (Other > processes can still send messages using gen_server:cast()) Why not do this processing in a handle_info? Trigger the calling of the handle_info call by sending a message to the gen_server. From antoine.koener@REDACTED Thu Mar 3 14:00:06 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Thu, 3 Mar 2011 14:00:06 +0100 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: Message-ID: <9375DC2B-92C1-4070-A1AA-C04B7E508065@gmail.com> On Mar 3, 2011, at 10:57 , Attila Rajmund Nohl wrote: > 2011/3/2, Nav : >> Hi, >> >> Is there a way to stop handle_cast(..) being called (i.e. stop >> gen_server from reading data from queue) for a while? >> >> I am trying to implement a logic, while processing, gen_server should >> not accept any messages and let messages stay in its queue. (Other >> processes can still send messages using gen_server:cast()) > > Why not do this processing in a handle_info? Trigger the calling of > the handle_info call by sending a message to the gen_server. > Was thinking about something like this: skip() -> skip(?MODULE). skip(Node) -> gen_server:cast(Node, skip); unskip() -> skip(?MODULE). unskip(Node) -> gen_server:cast(Node, unskip); [...] handle_cast(_Any, #state{skip=true} = State) -> {noreply, State}; handle_cast(skip, State) -> {noreply, State#state{skip=true}}; handle_cast(unskip, State) -> {noreply, State#state{skip=false}}; > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From seancribbs@REDACTED Thu Mar 3 15:23:29 2011 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 3 Mar 2011 09:23:29 -0500 Subject: [erlang-questions] Rubyists' #1 "wanna-learn" language? Maybe it's Erlang In-Reply-To: References: Message-ID: I'm a Rubyist and learned Erlang several years ago. No regrets! On Wed, Mar 2, 2011 at 6:22 AM, Michael Turner wrote: > And if so, perhaps by a wide margin. > > Interesting small-sample poll: > > ?http://weblog.jamisbuck.org/2011/2/22/programming-language-survey-results > > -michael turner > From attila.r.nohl@REDACTED Thu Mar 3 16:02:20 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 3 Mar 2011 16:02:20 +0100 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: <9375DC2B-92C1-4070-A1AA-C04B7E508065@gmail.com> References: <9375DC2B-92C1-4070-A1AA-C04B7E508065@gmail.com> Message-ID: 2011/3/3, Koener Antoine : > > On Mar 3, 2011, at 10:57 , Attila Rajmund Nohl wrote: > >> 2011/3/2, Nav : >>> Hi, >>> >>> Is there a way to stop handle_cast(..) being called (i.e. stop >>> gen_server from reading data from queue) for a while? >>> >>> I am trying to implement a logic, while processing, gen_server should >>> not accept any messages and let messages stay in its queue. (Other >>> processes can still send messages using gen_server:cast()) >> >> Why not do this processing in a handle_info? Trigger the calling of >> the handle_info call by sending a message to the gen_server. >> > > Was thinking about something like this: > [...] > > handle_cast(_Any, #state{skip=true} = State) -> > {noreply, State}; This way the "let messages stay in its queue" property would not hold, because this clause would just throw away any messages received during the "lock down" state. From jesper.louis.andersen@REDACTED Thu Mar 3 17:59:59 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 3 Mar 2011 17:59:59 +0100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: On Wed, Mar 2, 2011 at 19:02, Dhananjay Nene wrote: > While supervisors are meant to automatically restart failed processes, > there is one scenario I am as yet unable figure out which is the > idiomatic approach to implement crash recovery under the default OTP > scenarios. I have considered a solution, but being a relative newbie, > I am not sure if it is idiomatic erlang and if there are better > solutions. > > Question in short : If I have a supervisor which has a number of > dynamic children, how do I set up a mechanism where in case of a > complete system crash, all the dynamic children restart at the point > they were when the system (including the supervisor) crashed. Your only chance is to evacuate the state somewhere else or make all dangerous things happen in another process. You have to decide to which extent you want to persist the data on disk so you can read it in again. Personally, I would probably be looking at persisting the game state. The alternative solution also includes another machine that can take over the games. There is a blog post of mine, http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-and-crashes.html which I'll shamelessly plug because it gives some more ideas on how to think about this. -- J. From vances@REDACTED Thu Mar 3 18:28:07 2011 From: vances@REDACTED (Vance Shipley) Date: Thu, 3 Mar 2011 17:28:07 +0000 Subject: Erlang get_tcp:recv data length In-Reply-To: References: <201102231506406740978@its3.ch> <20110223192936.GD17893@hijacked.us> <201102241011387237768@its3.ch> <201102241425170794480@its3.ch> Message-ID: <20110303172806.GE3272@aluminum-2.local> Below is an example snipped from some code I use where the protocol on a TCP connection is ASCII line based with the Start-of-Transmission (STX) and End-of-Transmission (ETX) ASCII characters use to delinetated each "frame". In simple cases one frame (STX.......ETX) is received in one message. However when there are multiple messages sent quickly they could wind up in the same TCP packet. It could also be that with slow charater based writes one frame is received in multiple TCP packets. The code below is meant to buffer up the incoming feed and act only when a complete frame is received. -behaviour(gen_server). -define(STX, 2). -define(ETX, 3). handle_info({tcp, Socket, Data}, #state{socket = Socket} = State) -> handle_frame(Data, State); %% Parse the receive buffer for a framed record. %% handle_frame([?STX | Rest] = Data, State) -> case lists:splitwith(fun(?ETX) -> false; (_) -> true end, Rest) of {Record, [?ETX]} -> % handle received frame ... NewState = State#state{buffer = []}, {noreply, NewState}; {Record, [?ETX, ?STX | T]} -> % handle received frame ... NewState = State#state{buffer = []}, handle_frame([?STX | T], NewState); {_, []} -> NewState = State#state{buffer = Data}, {noreply, NewState} end; handle_frame([], State) -> {noreply, State}; handle_frame(Data, #state{buffer = [?STX | _]} = State) -> handle_frame(State#state.buffer ++ Data, State#state{buffer = []}); handle_frame([_ | T] = Data, #state{buffer = []} = State) -> handle_frame(T, State). -- -Vance From fritchie@REDACTED Thu Mar 3 18:47:05 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 03 Mar 2011 11:47:05 -0600 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: Message of "Wed, 02 Mar 2011 23:32:06 +0530." Message-ID: <81396.1299174425@snookles.snookles.com> Jesper Louis Anderson's blog entry makes a good argument for how to think about your program's state. Papers such as "Session State: Beyond Soft State" from 2004(*) talk about how to think of soft vs. hard state, though I'm sure there are probably papers from much earlier than that ... it's just one of the first to pop into my head. It's long been useful to me to consider which pieces of state data I cannot tolerate being lost and what state data is OK being transient (or recalculated based on hard state). -Scott (*) http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.10.2617 From bob@REDACTED Thu Mar 3 20:56:09 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 03 Mar 2011 19:56:09 +0000 Subject: Binaries Message-ID: <4D6FF259.4070107@bobcowdery.plus.com> One of the things I need to do is move large binaries of up to 4096 floats around between processes. As there are real-time constraints on processing this data I just wanted to check a few things. The docs say that large binaries are zero copy (presumable only in the same node). How exactly does this work in a lock free environment? 1> T=[1.0]. [1.0] 2> B=term_to_binary(T). <<131,108,0,0,0,1,99,49,46,48,48,48,48,48,48,48,48,48,48, 48,48,48,48,48,48,48,48,48,48,...>> 3> size(B). 39 The docs also say that binary data is efficient. Why does 1 float convert to 39 bytes? This ratio only reduces slightly with larger lists . If erlang holds floats internally in 64 bit format I don't understand why it needs 39 bytes. Bob From adam.kocoloski@REDACTED Thu Mar 3 22:18:28 2011 From: adam.kocoloski@REDACTED (Adam Kocoloski) Date: Thu, 3 Mar 2011 16:18:28 -0500 Subject: [erlang-questions] Binaries In-Reply-To: <4D6FF259.4070107@bobcowdery.plus.com> References: <4D6FF259.4070107@bobcowdery.plus.com> Message-ID: <245168A7-9433-4D77-867C-38D1E406F397@gmail.com> On Mar 3, 2011, at 2:56 PM, Bob Cowdery wrote: > One of the things I need to do is move large binaries of up to 4096 > floats around between processes. As there are real-time constraints on > processing this data I just wanted to check a few things. > > The docs say that large binaries are zero copy (presumable only in the > same node). How exactly does this work in a lock free environment? > > 1> T=[1.0]. > [1.0] > 2> B=term_to_binary(T). > <<131,108,0,0,0,1,99,49,46,48,48,48,48,48,48,48,48,48,48, > 48,48,48,48,48,48,48,48,48,48,...>> > 3> size(B). > 39 > > The docs also say that binary data is efficient. Why does 1 float > convert to 39 bytes? This ratio only reduces slightly with larger lists > . If erlang holds floats internally in 64 bit format I don't understand > why it needs 39 bytes. > > Bob Hi Bob, I think you're looking for the {minor_version, 1} option: http://erldocs.com/R14B01/erts/erlang.html?i=1&search=term_to#term_to_binary/2 1> byte_size(term_to_binary(1.0, [{minor_version, 1}])). 10 Regards, Adam From bob@REDACTED Thu Mar 3 22:21:02 2011 From: bob@REDACTED (Bob Ippolito) Date: Thu, 3 Mar 2011 13:21:02 -0800 Subject: [erlang-questions] Binaries In-Reply-To: <4D6FF259.4070107@bobcowdery.plus.com> References: <4D6FF259.4070107@bobcowdery.plus.com> Message-ID: Large binaries are reference counted. 1> byte_size(term_to_binary(1.0)). 33 2> byte_size(term_to_binary(1.0, [{minor_version, 1}])). 10 3> byte_size(<<1.0/float>>). 8 On Thu, Mar 3, 2011 at 11:56 AM, Bob Cowdery wrote: > One of the things I need to do is move large binaries of up to 4096 > floats around between processes. As there are real-time constraints on > processing this data I just wanted to check a few things. > > The docs say that large binaries are zero copy (presumable only in the > same node). How exactly does this work in a lock free environment? > > 1> T=[1.0]. > [1.0] > 2> B=term_to_binary(T). > <<131,108,0,0,0,1,99,49,46,48,48,48,48,48,48,48,48,48,48, > ?48,48,48,48,48,48,48,48,48,48,...>> > 3> size(B). > 39 > > The docs also say that binary data is efficient. Why does 1 float > convert to 39 bytes? This ratio only reduces slightly with larger lists > . If erlang holds floats internally in 64 bit format I don't understand > why it needs 39 bytes. > > Bob > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mihai@REDACTED Thu Mar 3 22:21:38 2011 From: mihai@REDACTED (Mihai Balea) Date: Thu, 3 Mar 2011 16:21:38 -0500 Subject: [erlang-questions] Binaries In-Reply-To: <4D6FF259.4070107@bobcowdery.plus.com> References: <4D6FF259.4070107@bobcowdery.plus.com> Message-ID: <62BF60D2-4A5F-4BC7-ADBA-4520E31029F4@hates.ms> On Mar 3, 2011, at 2:56 PM, Bob Cowdery wrote: > One of the things I need to do is move large binaries of up to 4096 > floats around between processes. As there are real-time constraints on > processing this data I just wanted to check a few things. > > The docs say that large binaries are zero copy (presumable only in the > same node). How exactly does this work in a lock free environment? Large binaries are stored in a separate heap and are garbage collected separately. A reference is passed around between processes (on the same node) > > 1> T=[1.0]. > [1.0] > 2> B=term_to_binary(T). > <<131,108,0,0,0,1,99,49,46,48,48,48,48,48,48,48,48,48,48, > 48,48,48,48,48,48,48,48,48,48,...>> > 3> size(B). > 39 > > The docs also say that binary data is efficient. Why does 1 float > convert to 39 bytes? This ratio only reduces slightly with larger lists > . If erlang holds floats internally in 64 bit format I don't understand > why it needs 39 bytes. First of all, in your example, T is a list containing one float element. Secondly, if you want to store a float in a binary, a better way would be 1> F = 1.0. 1.0 2> B = <>. <<63,240,0,0,0,0,0,0>> 3> size(B). 8 Mihai From max.lapshin@REDACTED Thu Mar 3 23:00:08 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 Mar 2011 01:00:08 +0300 Subject: [erlang-questions] Disallowing gen_server:handle_cast() for a while? In-Reply-To: References: <9375DC2B-92C1-4070-A1AA-C04B7E508065@gmail.com> Message-ID: This is exactly what I was speaking about: you need to use sys:suspend to lock down your process, do what you need and resume it back. All other ways will lead to parsing message queue From bob@REDACTED Thu Mar 3 23:04:21 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 03 Mar 2011 22:04:21 +0000 Subject: [erlang-questions] Binaries In-Reply-To: <245168A7-9433-4D77-867C-38D1E406F397@gmail.com> References: <4D6FF259.4070107@bobcowdery.plus.com> <245168A7-9433-4D77-867C-38D1E406F397@gmail.com> Message-ID: <4D701065.50308@bobcowdery.plus.com> On 03/03/2011 21:18, Adam Kocoloski wrote: > On Mar 3, 2011, at 2:56 PM, Bob Cowdery wrote: > >> One of the things I need to do is move large binaries of up to 4096 >> floats around between processes. As there are real-time constraints on >> processing this data I just wanted to check a few things. >> >> The docs say that large binaries are zero copy (presumable only in the >> same node). How exactly does this work in a lock free environment? >> >> 1> T=[1.0]. >> [1.0] >> 2> B=term_to_binary(T). >> <<131,108,0,0,0,1,99,49,46,48,48,48,48,48,48,48,48,48,48, >> 48,48,48,48,48,48,48,48,48,48,...>> >> 3> size(B). >> 39 >> >> The docs also say that binary data is efficient. Why does 1 float >> convert to 39 bytes? This ratio only reduces slightly with larger lists >> . If erlang holds floats internally in 64 bit format I don't understand >> why it needs 39 bytes. >> >> Bob > Hi Bob, I think you're looking for the {minor_version, 1} option: > > http://erldocs.com/R14B01/erts/erlang.html?i=1&search=term_to#term_to_binary/2 > > 1> byte_size(term_to_binary(1.0, [{minor_version, 1}])). > 10 > > Regards, Adam > ______ Hi Adam Thankyou, that's much more what I was expecting. The docs that install are only the application groups and the Basic/binary has none of this stuff in it. It seems one has to look in the ERTS manual. Rather confusing unless familiar with the system. Bob > __________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dizzyd@REDACTED Fri Mar 4 04:50:07 2011 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 3 Mar 2011 20:50:07 -0700 Subject: [erlang-questions] Why do people keep rebar binary in repos ? In-Reply-To: References: <20110302070147.GE30998@alumni.caltech.edu> Message-ID: On Wed, Mar 2, 2011 at 5:43 AM, Gleb Peregud wrote: >> Better would be if some form of semantic versioning were adopted and >> things tagged, and the version as part of the download. > > Take a look at http://semver.org/ Agreed and something we'll do. D. From fernando.benavides@REDACTED Fri Mar 4 20:53:06 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Fri, 04 Mar 2011 16:53:06 -0300 Subject: Dialyzer vs. Weird List Comprehensions Message-ID: <1299268386.3079.29.camel@army.local> Hi, I think this question is mostly for the man of the moment, Kostis Sagonas ;). But, if everybody else had run into this kind of things before, your help is appreciated. I have a quite complex module and at one point there is a function that uses list comprehensions over one of its parameters. That parameter, may eventually be an empty list (a hardcoded empty list - i.e. when know that in that particular case it'll be an empty list at compile time). So, when I try to run dialyzer on my module, it prints a warning about that when in my opinion, it shouldn't. To make things easier to understand, I've isolated the problem in this rather useless module: -module(listcomp). -export([sample/0]). -spec sample() -> []. sample() -> [L || L <- []]. Now, you can compile and run it: 1> c(listcomp). {ok,listcomp} 2> listcomp:sample(). [] It works fine, but if you try to dialyze it... $ dialyzer ebin/listcomp.beam [...] listcomp.erl:4: The pattern [L | _] can never match the type [] done in 0m0.31s done (warnings were emitted) First thought is "why are you doing a list comprehension on an empty list anyway?". But, for instance, the actual line in my module is something like this: all_friends(Buddies) -> [db:get_buddy(FN) || #buddy{friend_names = FNs} <- Buddies, FN <- FNs]. And it's called from a lot of places... one of those looks like this... all_friends([#buddy{name = "Jim (from 28 days later...)", ...}, #buddy{name = "Chuck Norris", ...}]) ...and you know... neither of those folks have any friends, so dialyzer complains. What do you think? Is it a bug? Should I modify my code not to use list comprehensions on that case? Cheers!! ________________________________________________________________________ Fernando Benavides fernando@REDACTED From mevans@REDACTED Fri Mar 4 22:55:00 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 4 Mar 2011 16:55:00 -0500 Subject: NIFs, what am I missing? Message-ID: The file is there: 9> filelib:is_file("/home/mevans/atis/idxavcspc.so"). true But will not load (won't load from the module either): 10> erlang:load_nif("/home/mevans/atis/idxavcspc",0). {error,{load_failed,"Failed to load NIF library: '/home/mevans/atis/idxavcspc.so: cannot open shared object file: No such file or directory'"}} I'm currently confused.... Thanks Matt Running: R14B01 From paul.joseph.davis@REDACTED Fri Mar 4 23:10:33 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Fri, 4 Mar 2011 17:10:33 -0500 Subject: [erlang-questions] NIFs, what am I missing? In-Reply-To: References: Message-ID: On Fri, Mar 4, 2011 at 4:55 PM, Evans, Matthew wrote: > The file is there: > 9> ?filelib:is_file("/home/mevans/atis/idxavcspc.so"). > true > > But will not load (won't load from the module either): > 10> ?erlang:load_nif("/home/mevans/atis/idxavcspc",0). > {error,{load_failed,"Failed to load NIF library: '/home/mevans/atis/idxavcspc.so: cannot open shared object file: No such file or directory'"}} > > I'm currently confused.... > > > Thanks > > Matt > > Running: R14B01 > Random guess from left field, perhaps the module loader isn't following a symlink in that path? From mevans@REDACTED Fri Mar 4 23:46:00 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 4 Mar 2011 17:46:00 -0500 Subject: [erlang-questions] NIFs, what am I missing? In-Reply-To: References: Message-ID: That wasn't it. I needed the -m32 option on gcc gcc -shared -fPIC -o idxavcspc.so idxavcspc.c -I /thirdPartyErlang/x86_32/usr/include/ -m32 -----Original Message----- From: Paul Davis [mailto:paul.joseph.davis@REDACTED] Sent: Friday, March 04, 2011 5:11 PM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] NIFs, what am I missing? On Fri, Mar 4, 2011 at 4:55 PM, Evans, Matthew wrote: > The file is there: > 9> ?filelib:is_file("/home/mevans/atis/idxavcspc.so"). > true > > But will not load (won't load from the module either): > 10> ?erlang:load_nif("/home/mevans/atis/idxavcspc",0). > {error,{load_failed,"Failed to load NIF library: '/home/mevans/atis/idxavcspc.so: cannot open shared object file: No such file or directory'"}} > > I'm currently confused.... > > > Thanks > > Matt > > Running: R14B01 > Random guess from left field, perhaps the module loader isn't following a symlink in that path? From robert.virding@REDACTED Sat Mar 5 00:23:12 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 4 Mar 2011 23:23:12 +0000 (GMT) Subject: [erlang-questions] Dialyzer vs. Weird List Comprehensions In-Reply-To: <2020389189.28251299280791765.JavaMail.root@zimbra> Message-ID: <1823923496.28271299280992661.JavaMail.root@zimbra> One guess is that might depend on the fact that dialyzer works on Core erlang, an internal compiler format, and at that stage list/binary comprehensions have already been expanded into local recursive functions. These comprehension functions will contain a clause which against the head of a list. Dialyzer will then detect that the comprehension function is only called with a [] and then warn that the pattern in this clause can never match. But this is just a guess. Robert -- Robert Virding, Erlang Solutions Ltd. ----- "Fernando Benavides" wrote: > Hi, I think this question is mostly for the man of the moment, Kostis > Sagonas ;). But, if everybody else had run into this kind of things > before, your help is appreciated. > I have a quite complex module and at one point there is a function > that > uses list comprehensions over one of its parameters. That parameter, > may > eventually be an empty list (a hardcoded empty list - i.e. when know > that in that particular case it'll be an empty list at compile time). > So, when I try to run dialyzer on my module, it prints a warning > about > that when in my opinion, it shouldn't. > To make things easier to understand, I've isolated the problem in > this > rather useless module: > > -module(listcomp). > -export([sample/0]). > -spec sample() -> []. > sample() -> [L || L <- []]. > > Now, you can compile and run it: > > 1> c(listcomp). > {ok,listcomp} > 2> listcomp:sample(). > [] > > It works fine, but if you try to dialyze it... > > $ dialyzer ebin/listcomp.beam > [...] > listcomp.erl:4: The pattern [L | _] can never match the type > [] > done in 0m0.31s > done (warnings were emitted) > > > First thought is "why are you doing a list comprehension on an empty > list anyway?". But, for instance, the actual line in my module is > something like this: > > all_friends(Buddies) -> [db:get_buddy(FN) || > #buddy{friend_names > = FNs} <- Buddies, FN <- FNs]. > > > And it's called from a lot of places... one of those looks like > this... > > all_friends([#buddy{name = "Jim (from 28 days later...)", > ...}, > #buddy{name = "Chuck Norris", ...}]) > > > ...and you know... neither of those folks have any friends, so > dialyzer > complains. > > What do you think? Is it a bug? Should I modify my code not to use > list > comprehensions on that case? > > Cheers!! > > > > ________________________________________________________________________ > Fernando > Benavides > > > fernando@REDACTED From comptekki@REDACTED Sat Mar 5 00:35:44 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 4 Mar 2011 16:35:44 -0700 Subject: erlang R14B01 on os x lion preview Message-ID: During make with R14B01 src (on os x lion preview) I get: ERL_TOP=/usr/local/src/otp_src_R14B01 PATH=/usr/local/src/otp_src_R14B01/bootstrap/bin:${PATH} \ make opt SECONDARY_BOOTSTRAP=true === Entering application hipe (cd ../main && make hipe.hrl) sed -e "s;%VSN%;3.7.8;" ../../hipe/main/hipe.hrl.src > ../../hipe/main/hipe.hrl erlc -W +debug_info +inline -o../ebin hipe_rtl.erl (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<39 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<42 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<6 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<12 bytes>>]},[{erlang,apply,2}]}\n" and it is just stuck there right now. Is it dead or still doing something? thx, -wes From comptekki@REDACTED Sat Mar 5 00:44:51 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 4 Mar 2011 16:44:51 -0700 Subject: erlang R14B01 on os x lion preview In-Reply-To: References: Message-ID: On Fri, Mar 4, 2011 at 4:35 PM, Wes James wrote: > During make with R14B01 src (on os x lion preview) I get: > > ? ? ? ? ?ERL_TOP=/usr/local/src/otp_src_R14B01 > PATH=/usr/local/src/otp_src_R14B01/bootstrap/bin:${PATH} \ > ? ? ? ? ? ? ? ?make opt SECONDARY_BOOTSTRAP=true > I did a ctrl-c and it did this at this point: BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution so it was in erlang doing some continued "make"... -wes From jameschurchman@REDACTED Sat Mar 5 02:19:38 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sat, 5 Mar 2011 01:19:38 +0000 Subject: [erlang-questions] Dialyzer vs. Weird List Comprehensions In-Reply-To: <1823923496.28271299280992661.JavaMail.root@zimbra> References: <1823923496.28271299280992661.JavaMail.root@zimbra> Message-ID: Is it not just the Dialyzer doing its job? I have been hit once or twice by list comprehensions, as they don't behave like the rest of erlang with regards to let it crash, which is very useful in some cases ... a pain in others eg i have a tuple, then loop over with a list comprehension : {item,Elem1,Elem2} Then in a different function / module i have a function that does say : [ Elem1 || {item,Elem1,Elem2} <- List ] later i modify the tuple ( / maybe other data structure) to {item,Elem1,Elem2,Elem3} and all is fine.. the code won't crash etc.. except it's totally broken. the match will never happen and i will always have an empty list this is great for user supplied data but makes using tuples with LC's a real annoyance ( i realise i could use a record but lots of extra records just to pack up an extra value or two from a return seems unnecessary) in my opinion maybe it should crash... or at least there should be a crashing and non crashing list comprehension apart from the record syntax it's the one other big problem i feel erlang has i would have : InputList = [ {item1,item2} , {item1} ] [ Item || { Item1 , Item2 } <- InputList ] %% would act like a list filter for anything but double tuple and will not crash (as it does at the moment) [ Item ||| { Item1 , Item2 } <- InputList ] %% notice the triple ||| : will crash on any non-matching input this would save quite a few "i gave my system some input and got an empty list out" type situations, as it would just crash i don't know the in's and out's but i think the Dialyzer is working out that your passing something that always results in the list comprehension producing an empty list.. which is not the usual use of a list comprehensions so attempting to warn you as this is ( i guess) a common fault in erlang programming. Can't you just not call the function in this case? ps does anybody think my suggestion is useful, and have they had the same experiences? i always hate the lists:map(fun) etc... it seems like unnecessary module redirection and is set to wreak havoc if i ever compile just that module with hype (well, i think it will :-) ) James On 4 Mar 2011, at 23:23, Robert Virding wrote: > One guess is that might depend on the fact that dialyzer works on Core erlang, an internal compiler format, and at that stage list/binary comprehensions have already been expanded into local recursive functions. These comprehension functions will contain a clause which against the head of a list. Dialyzer will then detect that the comprehension function is only called with a [] and then warn that the pattern in this clause can never match. > > But this is just a guess. > > Robert > > -- > Robert Virding, Erlang Solutions Ltd. > > ----- "Fernando Benavides" wrote: > >> Hi, I think this question is mostly for the man of the moment, Kostis >> Sagonas ;). But, if everybody else had run into this kind of things >> before, your help is appreciated. >> I have a quite complex module and at one point there is a function >> that >> uses list comprehensions over one of its parameters. That parameter, >> may >> eventually be an empty list (a hardcoded empty list - i.e. when know >> that in that particular case it'll be an empty list at compile time). >> So, when I try to run dialyzer on my module, it prints a warning >> about >> that when in my opinion, it shouldn't. >> To make things easier to understand, I've isolated the problem in >> this >> rather useless module: >> >> -module(listcomp). >> -export([sample/0]). >> -spec sample() -> []. >> sample() -> [L || L <- []]. >> >> Now, you can compile and run it: >> >> 1> c(listcomp). >> {ok,listcomp} >> 2> listcomp:sample(). >> [] >> >> It works fine, but if you try to dialyze it... >> >> $ dialyzer ebin/listcomp.beam >> [...] >> listcomp.erl:4: The pattern [L | _] can never match the type >> [] >> done in 0m0.31s >> done (warnings were emitted) >> >> >> First thought is "why are you doing a list comprehension on an empty >> list anyway?". But, for instance, the actual line in my module is >> something like this: >> >> all_friends(Buddies) -> [db:get_buddy(FN) || >> #buddy{friend_names >> = FNs} <- Buddies, FN <- FNs]. >> >> >> And it's called from a lot of places... one of those looks like >> this... >> >> all_friends([#buddy{name = "Jim (from 28 days later...)", >> ...}, >> #buddy{name = "Chuck Norris", ...}]) >> >> >> ...and you know... neither of those folks have any friends, so >> dialyzer >> complains. >> >> What do you think? Is it a bug? Should I modify my code not to use >> list >> comprehensions on that case? >> >> Cheers!! >> >> >> >> ________________________________________________________________________ >> Fernando >> Benavides >> >> >> fernando@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From brum76@REDACTED Sat Mar 5 02:53:40 2011 From: brum76@REDACTED (Radu Brumariu) Date: Fri, 4 Mar 2011 17:53:40 -0800 Subject: Erlang UG - Boston area Message-ID: Is there an Erlang User Group around Boston area ? If not, is anyone interested in starting one ? Thanks, Radu From yrashk@REDACTED Sat Mar 5 03:38:59 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 18:38:59 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <5393992.376.1299292739106.JavaMail.geo-discussion-forums@prih7> Let me guess, does you have cc pointing to clang or gcc? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sat Mar 5 03:38:59 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 18:38:59 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <5393992.376.1299292739106.JavaMail.geo-discussion-forums@prih7> Let me guess, does you have cc pointing to clang or gcc? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sat Mar 5 03:39:43 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 18:39:43 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <19931085.189.1299292783499.JavaMail.geo-discussion-forums@prcm18> Let me guess, do you have cc pointing to clang or gcc? -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sat Mar 5 03:39:43 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 18:39:43 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <19931085.189.1299292783499.JavaMail.geo-discussion-forums@prcm18> Let me guess, do you have cc pointing to clang or gcc? -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Sat Mar 5 04:15:11 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 4 Mar 2011 20:15:11 -0700 Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: <5393992.376.1299292739106.JavaMail.geo-discussion-forums@prih7> References: <5393992.376.1299292739106.JavaMail.geo-discussion-forums@prih7> Message-ID: On Fri, Mar 4, 2011 at 7:39 PM, Yurii Rashkovskii wrote: >Let me guess, do you have cc pointing to clang or gcc? I installed the xcode 4.1 developer preview. -wes From yrashk@REDACTED Sat Mar 5 04:26:35 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 19:26:35 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <29509329.1016.1299295595039.JavaMail.geo-discussion-forums@prnp16> It is a known issue, you need gcc (xcode 3), not clang. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sat Mar 5 04:26:35 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 4 Mar 2011 19:26:35 -0800 (PST) Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: Message-ID: <29509329.1016.1299295595039.JavaMail.geo-discussion-forums@prnp16> It is a known issue, you need gcc (xcode 3), not clang. -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Sat Mar 5 05:54:27 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 4 Mar 2011 21:54:27 -0700 Subject: [erlang-questions] erlang R14B01 on os x lion preview In-Reply-To: <29509329.1016.1299295595039.JavaMail.geo-discussion-forums@prnp16> References: <29509329.1016.1299295595039.JavaMail.geo-discussion-forums@prnp16> Message-ID: On Fri, Mar 4, 2011 at 8:26 PM, Yurii Rashkovskii wrote: > It is a known issue, you need gcc (xcode 3), not clang. > Thought so. It's a bit too new. thx, -wes From dhananjay.nene@REDACTED Sat Mar 5 06:49:12 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Sat, 5 Mar 2011 11:19:12 +0530 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: On Thu, Mar 3, 2011 at 10:29 PM, Jesper Louis Andersen wrote: > > On Wed, Mar 2, 2011 at 19:02, Dhananjay Nene wrote: > > Question in short : If I have a supervisor which has a number of > > dynamic children, how do I set up a mechanism where in case of a > > complete system crash, all the dynamic children restart at the point > > they were when the system (including the supervisor) crashed. > > Your only chance is to evacuate the state somewhere else or make all > dangerous things happen in another process. You have to decide to > which extent you want to persist the data on disk so you can read it > in again. Personally, I would probably be looking at persisting the > game state. The alternative solution also includes another machine > that can take over the games. > > There is a blog post of mine, > http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-and-crashes.html > which I'll shamelessly plug because it gives some more ideas on how to > think about this. > I read the post again, and again, and then again. While it did not give me a straight answer, it gave me the right framework to think through the answer, which is wonderful. I do have a follow up question, assuming the state is in the database to recover from, which option of the below is considered more idiomatic if at all there exists such a distinction a. Supervisor looks up state in db and feeds it to child during init b. Child self checks state in db on init. Thanks Dhananjay -- ----------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene From mevans@REDACTED Sat Mar 5 14:33:18 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 5 Mar 2011 08:33:18 -0500 Subject: [erlang-questions] Erlang UG - Boston area In-Reply-To: References: Message-ID: <2262F843-601B-4B7F-AF4E-0E5F624F8E38@verivue.com> There are quite a few Erlang companies in the Boston area. I think this would be a good idea. Maybe have a future Erlang conference here one year as an alternative to the west coast. Matt Sent from my iPhone On Mar 4, 2011, at 8:54 PM, "Radu Brumariu" wrote: > Is there an Erlang User Group around Boston area ? If not, is anyone > interested in starting one ? > > Thanks, > Radu > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From e_d_k@REDACTED Sat Mar 5 14:53:15 2011 From: e_d_k@REDACTED (Ed Keith) Date: Sat, 5 Mar 2011 05:53:15 -0800 (PST) Subject: [erlang-questions] Erlang UG - Boston area In-Reply-To: Message-ID: <305285.46427.qm@web120520.mail.ne1.yahoo.com> Sounds like a good idea to me. -EdK Ed Keith e_d_k@REDACTED Blog: edkeith.blogspot.com --- On Fri, 3/4/11, Radu Brumariu wrote: > From: Radu Brumariu > Subject: [erlang-questions] Erlang UG - Boston area > To: erlang-questions@REDACTED > Date: Friday, March 4, 2011, 8:53 PM > Is there an Erlang User Group around > Boston area ? If not, is anyone > interested in starting one ? > > Thanks, > Radu > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kostis@REDACTED Sat Mar 5 20:12:35 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 05 Mar 2011 21:12:35 +0200 Subject: [erlang-questions] Dialyzer vs. Weird List Comprehensions In-Reply-To: <1299268386.3079.29.camel@army.local> References: <1299268386.3079.29.camel@army.local> Message-ID: <4D728B23.2040900@cs.ntua.gr> Fernando Benavides wrote: > Hi, I think this question is mostly for the man of the moment, Kostis > Sagonas ;). But, if everybody else had run into this kind of things > before, your help is appreciated. > I have a quite complex module and at one point there is a function that > uses list comprehensions over one of its parameters. That parameter, may > eventually be an empty list (a hardcoded empty list - i.e. when know > that in that particular case it'll be an empty list at compile time). > So, when I try to run dialyzer on my module, it prints a warning about > that when in my opinion, it shouldn't. > To make things easier to understand, I've isolated the problem in this > rather useless module: > > -module(listcomp). > -export([sample/0]). > -spec sample() -> []. > sample() -> [L || L <- []]. > > It works fine, but if you try to dialyze it... > > $ dialyzer ebin/listcomp.beam > [...] > listcomp.erl:4: The pattern [L | _] can never match the type [] > done in 0m0.31s > done (warnings were emitted) Robert Virding has correctly replied that this warning exists because dialyzer works at the Core Erlang level where binary comprehensions have already been expanded to recursive functions by the Core Erlang translation which has one clause for when the list generator is empty and one for the cons case. But I want to point out something more else in your mail: > First thought is "why are you doing a list comprehension on an empty > list anyway?". But, for instance, the actual line in my module is > something like this: > > all_friends(Buddies) -> [db:get_buddy(FN) || #buddy{friend_names > = FNs} <- Buddies, FN <- FNs]. > > And it's called from a lot of places... one of those looks like this... > > all_friends([#buddy{name = "Jim (from 28 days later...)", ...}, > #buddy{name = "Chuck Norris", ...}]) > > ...and you know... neither of those folks have any friends, so dialyzer > complains. But what you described is not precise. It's not only in _one_ of these places that this happens, it's in *all* of them (within this module). I am sure of this, or there is some other problem in your code. Chuck Norris is even more sure of this. (*) Disprove him if you dare by sending the file that generates this warning! Cheers, Kostis (*) Fact: Chuck Norris can find and understand all bugs in an Erlang program without seeing the code. (for more facts see http://www.chucknorrisfacts.com/) From fernando.benavides@REDACTED Sun Mar 6 00:32:04 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Sat, 5 Mar 2011 20:32:04 -0300 Subject: [erlang-questions] Dialyzer vs. Weird List Comprehensions In-Reply-To: <4D728B23.2040900@cs.ntua.gr> References: <1299268386.3079.29.camel@army.local> <4D728B23.2040900@cs.ntua.gr> Message-ID: You (and of course Chuck) are right: The warning appears at the line where all_friends/1 is implemented. What I tried to say was that dialyzer stopped complaining about that if I removed that line because in every other place I was calling all_friends with lists that included at least one #buddy with a non empty list of friends. So, if I don't want to get that warning, I'll need to avoid using list comprehensions on empty lists. Thanks all of you (*) :) Fernando Benavides (*) Including Chuck On Sat, Mar 5, 2011 at 4:12 PM, Kostis Sagonas wrote: > Fernando Benavides wrote: > >> Hi, I think this question is mostly for the man of the moment, Kostis >> Sagonas ;). But, if everybody else had run into this kind of things >> before, your help is appreciated. >> I have a quite complex module and at one point there is a function that >> uses list comprehensions over one of its parameters. That parameter, may >> eventually be an empty list (a hardcoded empty list - i.e. when know >> that in that particular case it'll be an empty list at compile time). >> So, when I try to run dialyzer on my module, it prints a warning about >> that when in my opinion, it shouldn't. >> To make things easier to understand, I've isolated the problem in this >> rather useless module: >> >> -module(listcomp). >> -export([sample/0]). >> -spec sample() -> []. >> sample() -> [L || L <- []]. >> >> It works fine, but if you try to dialyze it... >> >> $ dialyzer ebin/listcomp.beam >> [...] >> listcomp.erl:4: The pattern [L | _] can never match the type [] >> done in 0m0.31s >> done (warnings were emitted) >> > > Robert Virding has correctly replied that this warning exists because > dialyzer works at the Core Erlang level where binary comprehensions have > already been expanded to recursive functions by the Core Erlang translation > which has one clause for when the list generator is empty and one for the > cons case. > > But I want to point out something more else in your mail: > > > First thought is "why are you doing a list comprehension on an empty >> list anyway?". But, for instance, the actual line in my module is >> something like this: >> >> all_friends(Buddies) -> [db:get_buddy(FN) || #buddy{friend_names >> = FNs} <- Buddies, FN <- FNs]. >> >> And it's called from a lot of places... one of those looks like this... >> >> all_friends([#buddy{name = "Jim (from 28 days later...)", ...}, >> #buddy{name = "Chuck Norris", ...}]) >> >> ...and you know... neither of those folks have any friends, so dialyzer >> complains. >> > > But what you described is not precise. It's not only in _one_ of these > places that this happens, it's in *all* of them (within this module). > I am sure of this, or there is some other problem in your code. Chuck > Norris is even more sure of this. (*) Disprove him if you dare by sending > the file that generates this warning! > > Cheers, > Kostis > > (*) Fact: Chuck Norris can find and understand all bugs in an Erlang > program without seeing the code. > (for more facts see http://www.chucknorrisfacts.com/) > -- *Fernando Benavides * From kaellis@REDACTED Sun Mar 6 06:54:33 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 6 Mar 2011 00:54:33 -0500 Subject: non-atomic nature of mnesia:dirty_update_counter Message-ID: Hello all, The description of mnesia:dirty_update_counter describes it as atomic. However I have found that two processes calling dirty_update_counter can obtain the same value. The description in the docs says: "mnesia:dirty_update_counter/3 is performed as an atomic operation despite the fact that it is not protected by a transaction." If I understand atomic to mean http://en.wikipedia.org/wiki/Linearizability, two processes calling dirty_update_counter can interfere with each other, as shown below. To repro, you'll have to set up the tables as shown in the mnesia:info output, then run the test, and compare the list of numbers spit out into the file. After running test_counter:run(5000,1) on each node simultaneously, and comparing the files, i get 2893 duplicates out of 5000. Although the counter has been incremented by 10000. I can't repro running on a single node with a large number of processes, so it seems atomic in that case. Would any of you call this atomic? Or was whoever wrote that doc thinking that the operation is atomic in the sense that its all-or-nothing. But I'd propose removing the word atomic from dirty_update_counter and replace it with a more crisply defined guarantee (if an error is returned, the counter is guaranteed not to have been incremented -- the ACID version of atomic), and to warn that multiple processes calling the method can interfere and obtain the same value. Ken -module(test_counter). -export([run/2,init/1]). -record(counter, {name, value}). init(NodeList) -> mnesia:create_table(counter, [{attributes, record_info(fields,counter)}, {disc_copies, NodeList}]). do_spawn(F,TSleep) -> timer:sleep(TSleep), spawn(F). run(NProcs,TSleep) -> Self = self(), CtrF = fun() -> Self ! mnesia:dirty_update_counter(counter,event_id,1) end, [ do_spawn(CtrF,TSleep) || _<-lists:duplicate(NProcs,0) ], CtrVals = gather(NProcs,[]), file:write_file("/tmp/counter_values",io_lib:format("~p",[CtrVals])). gather(0,L) -> L; gather(N,L) -> receive Ctr -> gather(N-1,[Ctr|L]) end. (test@REDACTED)44> mnesia:info(). ---> Processes holding locks <--- ---> Processes waiting for locks <--- ---> Participant transactions <--- ---> Coordinator transactions <--- ---> Uncertain transactions <--- ---> Active tables <--- counter : with 1 records occupying 310 words of mem schema : with 2 records occupying 520 words of mem ===> System info in version "4.4.16", debug level = none <=== opt_disc. Directory "/root/Mnesia.test@REDACTED" is used. use fallback at restart = false running db nodes = ['test@REDACTED','test@REDACTED'] stopped db nodes = [] master node tables = [] remote = [] ram_copies = [] disc_copies = [counter,schema] disc_only_copies = [] [{'test@REDACTED',disc_copies}, {'test@REDACTED',disc_copies}] = [schema,counter] 3 transactions committed, 0 aborted, 0 restarted, 144315 logged to disc 0 held locks, 0 in queue; 0 local transactions, 0 remote 0 transactions waits for other nodes: [] From ulf.wiger@REDACTED Sun Mar 6 08:19:22 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 6 Mar 2011 08:19:22 +0100 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: Message-ID: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> On 6 Mar 2011, at 06:54, Ken Ellis wrote: > After running test_counter:run(5000,1) on each node simultaneously, > and comparing the files, i get 2893 duplicates out of 5000. Although > the counter has been incremented by 10000. I can't repro running on a > single node with a large number of processes, so it seems atomic in > that case. It is exactly the single-node case that is atomic. The documentation should spell this out. The User Guide does say that you lose mnesia's atomicity and isolation properties if you use dirty operations, which does seem to contradict what is said about dirty_update_counter(), but what the Reference Manual (*and* User Guide) should say is that dirty_update_counter() is atomic only in a very limited context. The User Guide does say something else that is, strictly speaking, wrong: "It is not possible to have transaction protected updates of counter records." It is of course possible - they are only records after all. You can do an update counter inside a transaction through a combination of read() and write(). BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jwilberding@REDACTED Sun Mar 6 16:32:30 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Sun, 6 Mar 2011 10:32:30 -0500 Subject: [erlang-questions] Erlang UG - Boston area In-Reply-To: <2262F843-601B-4B7F-AF4E-0E5F624F8E38@verivue.com> References: <2262F843-601B-4B7F-AF4E-0E5F624F8E38@verivue.com> Message-ID: We are working on something for August in Boston. Stay tuned. Thanks! Jordan Wilberding On Sat, Mar 5, 2011 at 8:33 AM, Evans, Matthew wrote: > There are quite a few Erlang companies in the Boston area. I think this > would be a good idea. Maybe have a future Erlang conference here one year as > an alternative to the west coast. > > Matt > > Sent from my iPhone > > On Mar 4, 2011, at 8:54 PM, "Radu Brumariu" wrote: > > > Is there an Erlang User Group around Boston area ? If not, is anyone > > interested in starting one ? > > > > Thanks, > > Radu > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tristan.sloughter@REDACTED Sun Mar 6 16:41:05 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 09:41:05 -0600 Subject: Parse Transform with Macros Message-ID: I'd like to construct a parse transform that has a macro in the inserted code. I've had no problem using erl_scan:tokens and erl_parse:parse_form to construct the AST as long as no macros exist. Since macros are expanded before getting to this point it fails if in the string scanned has something like ?MODULE. I looked at epp_dodger and elsewhere but I can't seem to find something I could use to send in a string like: "init() -> ?MODULE." And have tokens I could use to then generate the AST. Is this possible? Thanks, Tristan From kaellis@REDACTED Sun Mar 6 17:12:23 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 6 Mar 2011 11:12:23 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Hi Ulf, Thanks for the clarification. I also went back to that User Guide and it does an admirable job of defining atomic as meaning the ACID all-or-nothing variety and not what for example you'd call atomic when incrementing counters within the linux kernel. My bad, forgot I was dealing with a database. So I think in that sense the dirty_update_counter is atomic across all nodes. I guess the question is when the users guide says atomicity is lost with dirty operations, what exactly is non-atomic about dirty_write or any dirty operation given that "Mnesia also ensures that all replicas of a table are updated if a dirty write operation is [successfully?] performed on a [any?] table". Isn't that the definition of all-or-nothing atomicity? Unless that statement is inaccurate, that would seem to mean all dirty_* operations are atomic, and that dirty is a reference to what an RDBMAS would call dirty reads, with an additional qualification that it can break isolation of transactions. Which by the way means there is no isolation guarantee for any transaction, since dirty operations compromise their isolation. (IMaybe filthy_* would be a better prefix :). But I suppose for now I'll take the statement about lack of atomicity as true, and that dirty writes can be only partially executed and can leave tables on different nodes in an inconsistent state. Ken On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger wrote: > > On 6 Mar 2011, at 06:54, Ken Ellis wrote: > > After running test_counter:run(5000,1) on each node simultaneously, > and comparing the files, i get 2893 duplicates out of 5000. ?Although > the counter has been incremented by 10000. ?I can't repro running on a > single node with a large number of processes, so it seems atomic in > that case. > > It is exactly the single-node case that is atomic. The documentation should > spell this out. > The User Guide does say that you lose mnesia's atomicity and isolation > properties if you use dirty operations, which does seem to contradict what > is said about dirty_update_counter(), but what the Reference Manual (*and* > User Guide) should say is that dirty_update_counter() is atomic only in a > very limited context. > The User Guide does say something else that is, strictly speaking, wrong: > "It is not possible to have transaction protected updates of counter > records." > It is of course possible - they are only records after all. You can do an > update counter inside a transaction through a combination of read() and > write(). > BR, > Ulf W > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From dgud@REDACTED Sun Mar 6 17:28:13 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Sun, 6 Mar 2011 17:28:13 +0100 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Nothing is promised with dirty_write don't use it. Dirty read may be ok if you know what you are doing but avoid dirty_write in multiple node system. /Dan On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis wrote: > Hi Ulf, > > Thanks for the clarification. ?I also went back to that User Guide and > it does an admirable job of defining atomic as meaning the ACID > all-or-nothing variety and not what for example you'd call atomic when > incrementing counters within the linux kernel. ?My bad, forgot I was > dealing with a database. ?So I think in that sense the > dirty_update_counter is atomic across all nodes. ?I guess the question > is when the users guide says atomicity is lost with dirty operations, > what exactly is non-atomic about dirty_write or any dirty operation > given that "Mnesia also ensures that all replicas of a table are > updated if a dirty write operation is [successfully?] performed on a > [any?] table". ?Isn't that the definition of all-or-nothing atomicity? > ?Unless that statement is inaccurate, that would seem to mean all > dirty_* operations are atomic, and that dirty is a reference to what > an RDBMAS would call dirty reads, with an additional qualification > that it can break isolation of transactions. ?Which by the way means > there is no isolation guarantee for any transaction, since dirty > operations compromise their isolation. ?(IMaybe filthy_* would be a > better prefix :). ?But I suppose for now I'll take the statement about > lack of atomicity as true, and that dirty writes can be only partially > executed and can leave tables on different nodes in an inconsistent > state. > > Ken > > > On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger > wrote: >> >> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >> >> After running test_counter:run(5000,1) on each node simultaneously, >> and comparing the files, i get 2893 duplicates out of 5000. ?Although >> the counter has been incremented by 10000. ?I can't repro running on a >> single node with a large number of processes, so it seems atomic in >> that case. >> >> It is exactly the single-node case that is atomic. The documentation should >> spell this out. >> The User Guide does say that you lose mnesia's atomicity and isolation >> properties if you use dirty operations, which does seem to contradict what >> is said about dirty_update_counter(), but what the Reference Manual (*and* >> User Guide) should say is that dirty_update_counter() is atomic only in a >> very limited context. >> The User Guide does say something else that is, strictly speaking, wrong: >> "It is not possible to have transaction protected updates of counter >> records." >> It is of course possible - they are only records after all. You can do an >> update counter inside a transaction through a combination of read() and >> write(). >> BR, >> Ulf W >> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kaellis@REDACTED Sun Mar 6 18:00:52 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 6 Mar 2011 12:00:52 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Amen, the whole dirty_* set of functions should be ripped out of the release. On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson wrote: > Nothing is promised with dirty_write don't use it. > > Dirty read may be ok if you know what you are doing > but avoid dirty_write in multiple node system. > > /Dan > > On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis wrote: >> Hi Ulf, >> >> Thanks for the clarification. ?I also went back to that User Guide and >> it does an admirable job of defining atomic as meaning the ACID >> all-or-nothing variety and not what for example you'd call atomic when >> incrementing counters within the linux kernel. ?My bad, forgot I was >> dealing with a database. ?So I think in that sense the >> dirty_update_counter is atomic across all nodes. ?I guess the question >> is when the users guide says atomicity is lost with dirty operations, >> what exactly is non-atomic about dirty_write or any dirty operation >> given that "Mnesia also ensures that all replicas of a table are >> updated if a dirty write operation is [successfully?] performed on a >> [any?] table". ?Isn't that the definition of all-or-nothing atomicity? >> ?Unless that statement is inaccurate, that would seem to mean all >> dirty_* operations are atomic, and that dirty is a reference to what >> an RDBMAS would call dirty reads, with an additional qualification >> that it can break isolation of transactions. ?Which by the way means >> there is no isolation guarantee for any transaction, since dirty >> operations compromise their isolation. ?(IMaybe filthy_* would be a >> better prefix :). ?But I suppose for now I'll take the statement about >> lack of atomicity as true, and that dirty writes can be only partially >> executed and can leave tables on different nodes in an inconsistent >> state. >> >> Ken >> >> >> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >> wrote: >>> >>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>> >>> After running test_counter:run(5000,1) on each node simultaneously, >>> and comparing the files, i get 2893 duplicates out of 5000. ?Although >>> the counter has been incremented by 10000. ?I can't repro running on a >>> single node with a large number of processes, so it seems atomic in >>> that case. >>> >>> It is exactly the single-node case that is atomic. The documentation should >>> spell this out. >>> The User Guide does say that you lose mnesia's atomicity and isolation >>> properties if you use dirty operations, which does seem to contradict what >>> is said about dirty_update_counter(), but what the Reference Manual (*and* >>> User Guide) should say is that dirty_update_counter() is atomic only in a >>> very limited context. >>> The User Guide does say something else that is, strictly speaking, wrong: >>> "It is not possible to have transaction protected updates of counter >>> records." >>> It is of course possible - they are only records after all. You can do an >>> update counter inside a transaction through a combination of read() and >>> write(). >>> BR, >>> Ulf W >>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From bob@REDACTED Sun Mar 6 18:07:47 2011 From: bob@REDACTED (Bob Ippolito) Date: Sun, 6 Mar 2011 09:07:47 -0800 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Well, they're actually quite useful (for performance) if you know what you're doing and the application can tolerate it. Often times the return value of a counter is not important. I suppose if they were ripped out, maybe fewer people would use Mnesia, which might be a net good :) On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis wrote: > Amen, the whole dirty_* set of functions should be ripped out of the release. > > > On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson wrote: >> Nothing is promised with dirty_write don't use it. >> >> Dirty read may be ok if you know what you are doing >> but avoid dirty_write in multiple node system. >> >> /Dan >> >> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis wrote: >>> Hi Ulf, >>> >>> Thanks for the clarification. ?I also went back to that User Guide and >>> it does an admirable job of defining atomic as meaning the ACID >>> all-or-nothing variety and not what for example you'd call atomic when >>> incrementing counters within the linux kernel. ?My bad, forgot I was >>> dealing with a database. ?So I think in that sense the >>> dirty_update_counter is atomic across all nodes. ?I guess the question >>> is when the users guide says atomicity is lost with dirty operations, >>> what exactly is non-atomic about dirty_write or any dirty operation >>> given that "Mnesia also ensures that all replicas of a table are >>> updated if a dirty write operation is [successfully?] performed on a >>> [any?] table". ?Isn't that the definition of all-or-nothing atomicity? >>> ?Unless that statement is inaccurate, that would seem to mean all >>> dirty_* operations are atomic, and that dirty is a reference to what >>> an RDBMAS would call dirty reads, with an additional qualification >>> that it can break isolation of transactions. ?Which by the way means >>> there is no isolation guarantee for any transaction, since dirty >>> operations compromise their isolation. ?(IMaybe filthy_* would be a >>> better prefix :). ?But I suppose for now I'll take the statement about >>> lack of atomicity as true, and that dirty writes can be only partially >>> executed and can leave tables on different nodes in an inconsistent >>> state. >>> >>> Ken >>> >>> >>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >>> wrote: >>>> >>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>>> >>>> After running test_counter:run(5000,1) on each node simultaneously, >>>> and comparing the files, i get 2893 duplicates out of 5000. ?Although >>>> the counter has been incremented by 10000. ?I can't repro running on a >>>> single node with a large number of processes, so it seems atomic in >>>> that case. >>>> >>>> It is exactly the single-node case that is atomic. The documentation should >>>> spell this out. >>>> The User Guide does say that you lose mnesia's atomicity and isolation >>>> properties if you use dirty operations, which does seem to contradict what >>>> is said about dirty_update_counter(), but what the Reference Manual (*and* >>>> User Guide) should say is that dirty_update_counter() is atomic only in a >>>> very limited context. >>>> The User Guide does say something else that is, strictly speaking, wrong: >>>> "It is not possible to have transaction protected updates of counter >>>> records." >>>> It is of course possible - they are only records after all. You can do an >>>> update counter inside a transaction through a combination of read() and >>>> write(). >>>> BR, >>>> Ulf W >>>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From emmiller@REDACTED Sun Mar 6 18:27:40 2011 From: emmiller@REDACTED (Evan Miller) Date: Sun, 6 Mar 2011 11:27:40 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Hi Tristan, Aleppo will apply preprocessor macros on tokens returned by erl_scan: https://github.com/evanmiller/aleppo I think it will fit your needs. Aleppo also lets you use -ifdef and the like inside of functions since it uses a proper grammar for the macro syntax. On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter wrote: > I'd like to construct a parse transform that has a macro in the inserted > code. I've had no problem using erl_scan:tokens and erl_parse:parse_form to > construct the AST as long as no macros exist. > > Since macros are expanded before getting to this point it fails if in the > string scanned has something like ?MODULE. I looked at epp_dodger and > elsewhere but I can't seem to find something I could use to send in a string > like: > > "init() -> ?MODULE." > > And have tokens I could use to then generate the AST. > > Is this possible? > > Thanks, > Tristan > -- Evan Miller http://www.evanmiller.org/ From tristan.sloughter@REDACTED Sun Mar 6 18:42:42 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 11:42:42 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Hey, this is great! But the result it gives me fails when sent to erl_parse: > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], "fun() -> ?MACHINE. ", 1). {done,{ok,[{'fun',1}, {'(',1}, {')',1}, {'->',1}, {'?',1}, {var,1,'MACHINE'}, {dot,1}], 1}, []} > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). {ok,[{'fun',1}, {'(',1}, {')',1}, {'->',1}, {atom,1,'BEAM'}, {dot,1}, {eof,0}]} > erl_parse:parse_form(NewTokens2). {error,{1,erl_parse,["syntax error before: ","'fun'"]}} Any ideas? Thanks. On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller wrote: > Hi Tristan, > > Aleppo will apply preprocessor macros on tokens returned by erl_scan: > > https://github.com/evanmiller/aleppo > > I think it will fit your needs. Aleppo also lets you use -ifdef and > the like inside of functions since it uses a proper grammar for the > macro syntax. > > On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter > wrote: > > I'd like to construct a parse transform that has a macro in the inserted > > code. I've had no problem using erl_scan:tokens and erl_parse:parse_form > to > > construct the AST as long as no macros exist. > > > > Since macros are expanded before getting to this point it fails if in the > > string scanned has something like ?MODULE. I looked at epp_dodger and > > elsewhere but I can't seem to find something I could use to send in a > string > > like: > > > > "init() -> ?MODULE." > > > > And have tokens I could use to then generate the AST. > > > > Is this possible? > > > > Thanks, > > Tristan > > > > > > -- > Evan Miller > http://www.evanmiller.org/ > From tristan.sloughter@REDACTED Sun Mar 6 18:46:14 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 11:46:14 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: OK, so maybe it helps if I wrote the fun correctly :). I fixed it to fun() -> ?MACHINE end. and I get: {error,{1,erl_parse,["syntax error before: ","'fun'"]}} On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Hey, this is great! But the result it gives me fails when sent to > erl_parse: > > > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], "fun() -> > ?MACHINE. ", 1). > {done,{ok,[{'fun',1}, > {'(',1}, > {')',1}, > {'->',1}, > {'?',1}, > {var,1,'MACHINE'}, > {dot,1}], > 1}, > []} > > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). > > {ok,[{'fun',1}, > {'(',1}, > {')',1}, > {'->',1}, > {atom,1,'BEAM'}, > {dot,1}, > {eof,0}]} > > erl_parse:parse_form(NewTokens2). > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > > Any ideas? > > Thanks. > > On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller wrote: > >> Hi Tristan, >> >> Aleppo will apply preprocessor macros on tokens returned by erl_scan: >> >> https://github.com/evanmiller/aleppo >> >> I think it will fit your needs. Aleppo also lets you use -ifdef and >> the like inside of functions since it uses a proper grammar for the >> macro syntax. >> >> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter >> wrote: >> > I'd like to construct a parse transform that has a macro in the inserted >> > code. I've had no problem using erl_scan:tokens and erl_parse:parse_form >> to >> > construct the AST as long as no macros exist. >> > >> > Since macros are expanded before getting to this point it fails if in >> the >> > string scanned has something like ?MODULE. I looked at epp_dodger and >> > elsewhere but I can't seem to find something I could use to send in a >> string >> > like: >> > >> > "init() -> ?MODULE." >> > >> > And have tokens I could use to then generate the AST. >> > >> > Is this possible? >> > >> > Thanks, >> > Tristan >> > >> >> >> >> -- >> Evan Miller >> http://www.evanmiller.org/ >> > > From kaellis@REDACTED Sun Mar 6 18:54:55 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 6 Mar 2011 12:54:55 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Yeah but to be blunt, that's two responses with "if you know what you're doing". I know what I'm doing, I just have no idea what Mnesia is doing. The documentation is inconsistent with itself and with Gudmundsson's remarks. Perhaps switch it off for any table spread across multiple nodes if no guarantees can be made in that case? On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito wrote: > Well, they're actually quite useful (for performance) if you know what > you're doing and the application can tolerate it. Often times the > return value of a counter is not important. I suppose if they were > ripped out, maybe fewer people would use Mnesia, which might be a net > good :) > > On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis wrote: >> Amen, the whole dirty_* set of functions should be ripped out of the release. >> >> >> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson wrote: >>> Nothing is promised with dirty_write don't use it. >>> >>> Dirty read may be ok if you know what you are doing >>> but avoid dirty_write in multiple node system. >>> >>> /Dan >>> >>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis wrote: >>>> Hi Ulf, >>>> >>>> Thanks for the clarification. ?I also went back to that User Guide and >>>> it does an admirable job of defining atomic as meaning the ACID >>>> all-or-nothing variety and not what for example you'd call atomic when >>>> incrementing counters within the linux kernel. ?My bad, forgot I was >>>> dealing with a database. ?So I think in that sense the >>>> dirty_update_counter is atomic across all nodes. ?I guess the question >>>> is when the users guide says atomicity is lost with dirty operations, >>>> what exactly is non-atomic about dirty_write or any dirty operation >>>> given that "Mnesia also ensures that all replicas of a table are >>>> updated if a dirty write operation is [successfully?] performed on a >>>> [any?] table". ?Isn't that the definition of all-or-nothing atomicity? >>>> ?Unless that statement is inaccurate, that would seem to mean all >>>> dirty_* operations are atomic, and that dirty is a reference to what >>>> an RDBMAS would call dirty reads, with an additional qualification >>>> that it can break isolation of transactions. ?Which by the way means >>>> there is no isolation guarantee for any transaction, since dirty >>>> operations compromise their isolation. ?(IMaybe filthy_* would be a >>>> better prefix :). ?But I suppose for now I'll take the statement about >>>> lack of atomicity as true, and that dirty writes can be only partially >>>> executed and can leave tables on different nodes in an inconsistent >>>> state. >>>> >>>> Ken >>>> >>>> >>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >>>> wrote: >>>>> >>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>>>> >>>>> After running test_counter:run(5000,1) on each node simultaneously, >>>>> and comparing the files, i get 2893 duplicates out of 5000. ?Although >>>>> the counter has been incremented by 10000. ?I can't repro running on a >>>>> single node with a large number of processes, so it seems atomic in >>>>> that case. >>>>> >>>>> It is exactly the single-node case that is atomic. The documentation should >>>>> spell this out. >>>>> The User Guide does say that you lose mnesia's atomicity and isolation >>>>> properties if you use dirty operations, which does seem to contradict what >>>>> is said about dirty_update_counter(), but what the Reference Manual (*and* >>>>> User Guide) should say is that dirty_update_counter() is atomic only in a >>>>> very limited context. >>>>> The User Guide does say something else that is, strictly speaking, wrong: >>>>> "It is not possible to have transaction protected updates of counter >>>>> records." >>>>> It is of course possible - they are only records after all. You can do an >>>>> update counter inside a transaction through a combination of read() and >>>>> write(). >>>>> BR, >>>>> Ulf W >>>>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >>>>> http://erlang-solutions.com >>>>> >>>>> >>>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From bob@REDACTED Sun Mar 6 19:01:24 2011 From: bob@REDACTED (Bob Ippolito) Date: Sun, 6 Mar 2011 10:01:24 -0800 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: Well, I'm sure you do know what you're doing in general, but not with Mnesia. Mnesia is quirky. You will probably eventually regret using Mnesia. We try and avoid it as much as possible, because of its inability to do the right thing with network partitions and the dets limitations for disk tables. If you can afford to, I'd look at using something else instead. On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: > Yeah but to be blunt, that's two responses with "if you know what > you're doing". ?I know what I'm doing, I just have no idea what Mnesia > is doing. ?The documentation is inconsistent with itself and with > Gudmundsson's remarks. ?Perhaps switch it off for any table spread > across multiple nodes if no guarantees can be made in that case? > > > On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito wrote: >> Well, they're actually quite useful (for performance) if you know what >> you're doing and the application can tolerate it. Often times the >> return value of a counter is not important. I suppose if they were >> ripped out, maybe fewer people would use Mnesia, which might be a net >> good :) >> >> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis wrote: >>> Amen, the whole dirty_* set of functions should be ripped out of the release. >>> >>> >>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson wrote: >>>> Nothing is promised with dirty_write don't use it. >>>> >>>> Dirty read may be ok if you know what you are doing >>>> but avoid dirty_write in multiple node system. >>>> >>>> /Dan >>>> >>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis wrote: >>>>> Hi Ulf, >>>>> >>>>> Thanks for the clarification. ?I also went back to that User Guide and >>>>> it does an admirable job of defining atomic as meaning the ACID >>>>> all-or-nothing variety and not what for example you'd call atomic when >>>>> incrementing counters within the linux kernel. ?My bad, forgot I was >>>>> dealing with a database. ?So I think in that sense the >>>>> dirty_update_counter is atomic across all nodes. ?I guess the question >>>>> is when the users guide says atomicity is lost with dirty operations, >>>>> what exactly is non-atomic about dirty_write or any dirty operation >>>>> given that "Mnesia also ensures that all replicas of a table are >>>>> updated if a dirty write operation is [successfully?] performed on a >>>>> [any?] table". ?Isn't that the definition of all-or-nothing atomicity? >>>>> ?Unless that statement is inaccurate, that would seem to mean all >>>>> dirty_* operations are atomic, and that dirty is a reference to what >>>>> an RDBMAS would call dirty reads, with an additional qualification >>>>> that it can break isolation of transactions. ?Which by the way means >>>>> there is no isolation guarantee for any transaction, since dirty >>>>> operations compromise their isolation. ?(IMaybe filthy_* would be a >>>>> better prefix :). ?But I suppose for now I'll take the statement about >>>>> lack of atomicity as true, and that dirty writes can be only partially >>>>> executed and can leave tables on different nodes in an inconsistent >>>>> state. >>>>> >>>>> Ken >>>>> >>>>> >>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >>>>> wrote: >>>>>> >>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>>>>> >>>>>> After running test_counter:run(5000,1) on each node simultaneously, >>>>>> and comparing the files, i get 2893 duplicates out of 5000. ?Although >>>>>> the counter has been incremented by 10000. ?I can't repro running on a >>>>>> single node with a large number of processes, so it seems atomic in >>>>>> that case. >>>>>> >>>>>> It is exactly the single-node case that is atomic. The documentation should >>>>>> spell this out. >>>>>> The User Guide does say that you lose mnesia's atomicity and isolation >>>>>> properties if you use dirty operations, which does seem to contradict what >>>>>> is said about dirty_update_counter(), but what the Reference Manual (*and* >>>>>> User Guide) should say is that dirty_update_counter() is atomic only in a >>>>>> very limited context. >>>>>> The User Guide does say something else that is, strictly speaking, wrong: >>>>>> "It is not possible to have transaction protected updates of counter >>>>>> records." >>>>>> It is of course possible - they are only records after all. You can do an >>>>>> update counter inside a transaction through a combination of read() and >>>>>> write(). >>>>>> BR, >>>>>> Ulf W >>>>>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >>>>>> http://erlang-solutions.com >>>>>> >>>>>> >>>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> > From emmiller@REDACTED Sun Mar 6 19:41:56 2011 From: emmiller@REDACTED (Evan Miller) Date: Sun, 6 Mar 2011 12:41:56 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: IIRC erl_parse needs a full module as input, i.e. -module() at the top and all Erlang expressions inside of named functions. On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter wrote: > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to fun() > -> ?MACHINE end. and I get: > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter > wrote: >> >> Hey, this is great! But the result it gives me fails when sent to >> erl_parse: >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], "fun() -> >> > ?MACHINE. ", 1). >> {done,{ok,[{'fun',1}, >> ?? ? ? ? ? {'(',1}, >> ?? ? ? ? ? {')',1}, >> ?? ? ? ? ? {'->',1}, >> ?? ? ? ? ? {'?',1}, >> ?? ? ? ? ? {var,1,'MACHINE'}, >> ?? ? ? ? ? {dot,1}], >> ?? ? ? ? ?1}, >> ?? ? ?[]} >> > ?{ok, NewTokens2} = aleppo:process_tokens(Tokens2). >> > >> {ok,[{'fun',1}, >> ?? ? {'(',1}, >> ?? ? {')',1}, >> ?? ? {'->',1}, >> ?? ? {atom,1,'BEAM'}, >> ?? ? {dot,1}, >> ?? ? {eof,0}]} >> > erl_parse:parse_form(NewTokens2). >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >> Any ideas? >> Thanks. >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller wrote: >>> >>> Hi Tristan, >>> >>> Aleppo will apply preprocessor macros on tokens returned by erl_scan: >>> >>> https://github.com/evanmiller/aleppo >>> >>> I think it will fit your needs. Aleppo also lets you use -ifdef and >>> the like inside of functions since it uses a proper grammar for the >>> macro syntax. >>> >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter >>> wrote: >>> > I'd like to construct a parse transform that has a macro in the >>> > inserted >>> > code. I've had no problem using erl_scan:tokens and >>> > erl_parse:parse_form to >>> > construct the AST as long as no macros exist. >>> > >>> > Since macros are expanded before getting to this point it fails if in >>> > the >>> > string scanned has something like ?MODULE. I looked at epp_dodger and >>> > elsewhere but I can't seem to find something I could use to send in a >>> > string >>> > like: >>> > >>> > "init() -> ?MODULE." >>> > >>> > And have tokens I could use to then generate the AST. >>> > >>> > Is this possible? >>> > >>> > Thanks, >>> > Tristan >>> > >>> >>> >>> >>> -- >>> Evan Miller >>> http://www.evanmiller.org/ >> > > -- Evan Miller http://www.evanmiller.org/ From tristan.sloughter@REDACTED Sun Mar 6 19:46:07 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 12:46:07 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: This was working for me with erl_scan and erl_parse. The error I'm getting is: {{badmatch, {error, {0,erl_parse, ["syntax error before: ", "eof"]}}}, I got the code for what I'm doing by reading: http://chlorophil.blogspot.com/2007/04/erlang-macro-processor-v2-part-v.html But then I wanted to use a macro in what I am added to the module. So I've modified 'ast_reversed_results' to: .... String -> {done,{ok,Tokens,LineEnd},StringRest} = erl_scan:tokens([], String, LineStart), {ok, NewTokens} = aleppo:process_tokens(Tokens), {ok, AST} = erl_parse:parse_form(NewTokens), .... But this fails with the above error. Maybe I'm doing something else wrong, I'll keep working on it. Thanks, Tristan On Sun, Mar 6, 2011 at 12:41 PM, Evan Miller wrote: > IIRC erl_parse needs a full module as input, i.e. -module() at the top > and all Erlang expressions inside of named functions. > > On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter > wrote: > > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to > fun() > > -> ?MACHINE end. and I get: > > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > > > > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter > > wrote: > >> > >> Hey, this is great! But the result it gives me fails when sent to > >> erl_parse: > >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], "fun() > -> > >> > ?MACHINE. ", 1). > >> {done,{ok,[{'fun',1}, > >> {'(',1}, > >> {')',1}, > >> {'->',1}, > >> {'?',1}, > >> {var,1,'MACHINE'}, > >> {dot,1}], > >> 1}, > >> []} > >> > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). > >> > > >> {ok,[{'fun',1}, > >> {'(',1}, > >> {')',1}, > >> {'->',1}, > >> {atom,1,'BEAM'}, > >> {dot,1}, > >> {eof,0}]} > >> > erl_parse:parse_form(NewTokens2). > >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > >> Any ideas? > >> Thanks. > >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller > wrote: > >>> > >>> Hi Tristan, > >>> > >>> Aleppo will apply preprocessor macros on tokens returned by erl_scan: > >>> > >>> https://github.com/evanmiller/aleppo > >>> > >>> I think it will fit your needs. Aleppo also lets you use -ifdef and > >>> the like inside of functions since it uses a proper grammar for the > >>> macro syntax. > >>> > >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter > >>> wrote: > >>> > I'd like to construct a parse transform that has a macro in the > >>> > inserted > >>> > code. I've had no problem using erl_scan:tokens and > >>> > erl_parse:parse_form to > >>> > construct the AST as long as no macros exist. > >>> > > >>> > Since macros are expanded before getting to this point it fails if in > >>> > the > >>> > string scanned has something like ?MODULE. I looked at epp_dodger and > >>> > elsewhere but I can't seem to find something I could use to send in a > >>> > string > >>> > like: > >>> > > >>> > "init() -> ?MODULE." > >>> > > >>> > And have tokens I could use to then generate the AST. > >>> > > >>> > Is this possible? > >>> > > >>> > Thanks, > >>> > Tristan > >>> > > >>> > >>> > >>> > >>> -- > >>> Evan Miller > >>> http://www.evanmiller.org/ > >> > > > > > > > > -- > Evan Miller > http://www.evanmiller.org/ > From emmiller@REDACTED Sun Mar 6 19:51:26 2011 From: emmiller@REDACTED (Evan Miller) Date: Sun, 6 Mar 2011 12:51:26 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Aleppo adds an "eof" token at the end if one is not present. Perhaps try removing it before passing to erl_parse? It's a new "feature" and perhaps a cause of bugs. On Sun, Mar 6, 2011 at 12:46 PM, Tristan Sloughter wrote: > This was working for me with erl_scan and erl_parse. The error I'm getting > is: > {{badmatch, > ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {error, > ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{0,erl_parse, > ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ["syntax error before: ", > ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?"eof"]}}}, > I got the code for what I'm doing by > reading:?http://chlorophil.blogspot.com/2007/04/erlang-macro-processor-v2-part-v.html > But then I wanted to use a macro in what I am added to the module. So I've > modified 'ast_reversed_results' to: > .... > ?? ? ? ?String -> > ?? ? ? ? ? ?{done,{ok,Tokens,LineEnd},StringRest} = > ?? ? ? ? ? ? ? ?erl_scan:tokens([], String, LineStart), > ?? ? ? ? ? ?{ok, NewTokens} = aleppo:process_tokens(Tokens), > ?? ? ? ? ? ?{ok, AST} = erl_parse:parse_form(NewTokens), > .... > But this fails with the above error. > Maybe I'm doing something else wrong, I'll keep working on it. > Thanks, > Tristan > On Sun, Mar 6, 2011 at 12:41 PM, Evan Miller wrote: >> >> IIRC erl_parse needs a full module as input, i.e. -module() at the top >> and all Erlang expressions inside of named functions. >> >> On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter >> wrote: >> > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to >> > fun() >> > -> ?MACHINE end. and I get: >> > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >> > >> > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter >> > wrote: >> >> >> >> Hey, this is great! But the result it gives me fails when sent to >> >> erl_parse: >> >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], "fun() >> >> > -> >> >> > ?MACHINE. ", 1). >> >> {done,{ok,[{'fun',1}, >> >> ?? ? ? ? ? {'(',1}, >> >> ?? ? ? ? ? {')',1}, >> >> ?? ? ? ? ? {'->',1}, >> >> ?? ? ? ? ? {'?',1}, >> >> ?? ? ? ? ? {var,1,'MACHINE'}, >> >> ?? ? ? ? ? {dot,1}], >> >> ?? ? ? ? ?1}, >> >> ?? ? ?[]} >> >> > ?{ok, NewTokens2} = aleppo:process_tokens(Tokens2). >> >> > >> >> {ok,[{'fun',1}, >> >> ?? ? {'(',1}, >> >> ?? ? {')',1}, >> >> ?? ? {'->',1}, >> >> ?? ? {atom,1,'BEAM'}, >> >> ?? ? {dot,1}, >> >> ?? ? {eof,0}]} >> >> > erl_parse:parse_form(NewTokens2). >> >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >> >> Any ideas? >> >> Thanks. >> >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller >> >> wrote: >> >>> >> >>> Hi Tristan, >> >>> >> >>> Aleppo will apply preprocessor macros on tokens returned by erl_scan: >> >>> >> >>> https://github.com/evanmiller/aleppo >> >>> >> >>> I think it will fit your needs. Aleppo also lets you use -ifdef and >> >>> the like inside of functions since it uses a proper grammar for the >> >>> macro syntax. >> >>> >> >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter >> >>> wrote: >> >>> > I'd like to construct a parse transform that has a macro in the >> >>> > inserted >> >>> > code. I've had no problem using erl_scan:tokens and >> >>> > erl_parse:parse_form to >> >>> > construct the AST as long as no macros exist. >> >>> > >> >>> > Since macros are expanded before getting to this point it fails if >> >>> > in >> >>> > the >> >>> > string scanned has something like ?MODULE. I looked at epp_dodger >> >>> > and >> >>> > elsewhere but I can't seem to find something I could use to send in >> >>> > a >> >>> > string >> >>> > like: >> >>> > >> >>> > "init() -> ?MODULE." >> >>> > >> >>> > And have tokens I could use to then generate the AST. >> >>> > >> >>> > Is this possible? >> >>> > >> >>> > Thanks, >> >>> > Tristan >> >>> > >> >>> >> >>> >> >>> >> >>> -- >> >>> Evan Miller >> >>> http://www.evanmiller.org/ >> >> >> > >> > >> >> >> >> -- >> Evan Miller >> http://www.evanmiller.org/ > > -- Evan Miller http://www.evanmiller.org/ From tristan.sloughter@REDACTED Sun Mar 6 20:24:45 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 13:24:45 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Sweet, that worked, thanks! Tristan On Sun, Mar 6, 2011 at 12:51 PM, Evan Miller wrote: > Aleppo adds an "eof" token at the end if one is not present. Perhaps > try removing it before passing to erl_parse? It's a new "feature" and > perhaps a cause of bugs. > > On Sun, Mar 6, 2011 at 12:46 PM, Tristan Sloughter > wrote: > > This was working for me with erl_scan and erl_parse. The error I'm > getting > > is: > > {{badmatch, > > {error, > > {0,erl_parse, > > ["syntax error before: > ", > > "eof"]}}}, > > I got the code for what I'm doing by > > reading: > http://chlorophil.blogspot.com/2007/04/erlang-macro-processor-v2-part-v.html > > But then I wanted to use a macro in what I am added to the module. So > I've > > modified 'ast_reversed_results' to: > > .... > > String -> > > {done,{ok,Tokens,LineEnd},StringRest} = > > erl_scan:tokens([], String, LineStart), > > {ok, NewTokens} = aleppo:process_tokens(Tokens), > > {ok, AST} = erl_parse:parse_form(NewTokens), > > .... > > But this fails with the above error. > > Maybe I'm doing something else wrong, I'll keep working on it. > > Thanks, > > Tristan > > On Sun, Mar 6, 2011 at 12:41 PM, Evan Miller wrote: > >> > >> IIRC erl_parse needs a full module as input, i.e. -module() at the top > >> and all Erlang expressions inside of named functions. > >> > >> On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter > >> wrote: > >> > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to > >> > fun() > >> > -> ?MACHINE end. and I get: > >> > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > >> > > >> > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter > >> > wrote: > >> >> > >> >> Hey, this is great! But the result it gives me fails when sent to > >> >> erl_parse: > >> >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], > "fun() > >> >> > -> > >> >> > ?MACHINE. ", 1). > >> >> {done,{ok,[{'fun',1}, > >> >> {'(',1}, > >> >> {')',1}, > >> >> {'->',1}, > >> >> {'?',1}, > >> >> {var,1,'MACHINE'}, > >> >> {dot,1}], > >> >> 1}, > >> >> []} > >> >> > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). > >> >> > > >> >> {ok,[{'fun',1}, > >> >> {'(',1}, > >> >> {')',1}, > >> >> {'->',1}, > >> >> {atom,1,'BEAM'}, > >> >> {dot,1}, > >> >> {eof,0}]} > >> >> > erl_parse:parse_form(NewTokens2). > >> >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} > >> >> Any ideas? > >> >> Thanks. > >> >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller > >> >> wrote: > >> >>> > >> >>> Hi Tristan, > >> >>> > >> >>> Aleppo will apply preprocessor macros on tokens returned by > erl_scan: > >> >>> > >> >>> https://github.com/evanmiller/aleppo > >> >>> > >> >>> I think it will fit your needs. Aleppo also lets you use -ifdef and > >> >>> the like inside of functions since it uses a proper grammar for the > >> >>> macro syntax. > >> >>> > >> >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter > >> >>> wrote: > >> >>> > I'd like to construct a parse transform that has a macro in the > >> >>> > inserted > >> >>> > code. I've had no problem using erl_scan:tokens and > >> >>> > erl_parse:parse_form to > >> >>> > construct the AST as long as no macros exist. > >> >>> > > >> >>> > Since macros are expanded before getting to this point it fails if > >> >>> > in > >> >>> > the > >> >>> > string scanned has something like ?MODULE. I looked at epp_dodger > >> >>> > and > >> >>> > elsewhere but I can't seem to find something I could use to send > in > >> >>> > a > >> >>> > string > >> >>> > like: > >> >>> > > >> >>> > "init() -> ?MODULE." > >> >>> > > >> >>> > And have tokens I could use to then generate the AST. > >> >>> > > >> >>> > Is this possible? > >> >>> > > >> >>> > Thanks, > >> >>> > Tristan > >> >>> > > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Evan Miller > >> >>> http://www.evanmiller.org/ > >> >> > >> > > >> > > >> > >> > >> > >> -- > >> Evan Miller > >> http://www.evanmiller.org/ > > > > > > > > -- > Evan Miller > http://www.evanmiller.org/ > From tristan.sloughter@REDACTED Sun Mar 6 20:44:52 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 13:44:52 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Is the file option so it is able to expand macros by providing a include file as an option? This isn't working and neither is if they are included in the module that I'm running the process tokens from. So this is only working if i just use the macro ?MACHINE which I guess always exists... Any other and it is unable to find it: {function_clause, [{lists,reverse, [{badarg, [{dict,fetch, ['BASE_MODULE', {dict,1,16,16,8,80,48, {[],[],[],[],[],[],[], [],[],[],[],[],[],[], [],[]}, {{[],[],[],[],[], [['MACHINE', {atom,1,'BEAM'}]], [],[],[],[],[],[], [],[],[],[]}}}]}, {aleppo,process_tree,3}, Tristan On Sun, Mar 6, 2011 at 1:24 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Sweet, that worked, thanks! > > Tristan > > > On Sun, Mar 6, 2011 at 12:51 PM, Evan Miller wrote: > >> Aleppo adds an "eof" token at the end if one is not present. Perhaps >> try removing it before passing to erl_parse? It's a new "feature" and >> perhaps a cause of bugs. >> >> On Sun, Mar 6, 2011 at 12:46 PM, Tristan Sloughter >> wrote: >> > This was working for me with erl_scan and erl_parse. The error I'm >> getting >> > is: >> > {{badmatch, >> > {error, >> > {0,erl_parse, >> > ["syntax error >> before: ", >> > "eof"]}}}, >> > I got the code for what I'm doing by >> > reading: >> http://chlorophil.blogspot.com/2007/04/erlang-macro-processor-v2-part-v.html >> > But then I wanted to use a macro in what I am added to the module. So >> I've >> > modified 'ast_reversed_results' to: >> > .... >> > String -> >> > {done,{ok,Tokens,LineEnd},StringRest} = >> > erl_scan:tokens([], String, LineStart), >> > {ok, NewTokens} = aleppo:process_tokens(Tokens), >> > {ok, AST} = erl_parse:parse_form(NewTokens), >> > .... >> > But this fails with the above error. >> > Maybe I'm doing something else wrong, I'll keep working on it. >> > Thanks, >> > Tristan >> > On Sun, Mar 6, 2011 at 12:41 PM, Evan Miller >> wrote: >> >> >> >> IIRC erl_parse needs a full module as input, i.e. -module() at the top >> >> and all Erlang expressions inside of named functions. >> >> >> >> On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter >> >> wrote: >> >> > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to >> >> > fun() >> >> > -> ?MACHINE end. and I get: >> >> > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >> >> > >> >> > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter >> >> > wrote: >> >> >> >> >> >> Hey, this is great! But the result it gives me fails when sent to >> >> >> erl_parse: >> >> >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], >> "fun() >> >> >> > -> >> >> >> > ?MACHINE. ", 1). >> >> >> {done,{ok,[{'fun',1}, >> >> >> {'(',1}, >> >> >> {')',1}, >> >> >> {'->',1}, >> >> >> {'?',1}, >> >> >> {var,1,'MACHINE'}, >> >> >> {dot,1}], >> >> >> 1}, >> >> >> []} >> >> >> > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). >> >> >> > >> >> >> {ok,[{'fun',1}, >> >> >> {'(',1}, >> >> >> {')',1}, >> >> >> {'->',1}, >> >> >> {atom,1,'BEAM'}, >> >> >> {dot,1}, >> >> >> {eof,0}]} >> >> >> > erl_parse:parse_form(NewTokens2). >> >> >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >> >> >> Any ideas? >> >> >> Thanks. >> >> >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller >> >> >> wrote: >> >> >>> >> >> >>> Hi Tristan, >> >> >>> >> >> >>> Aleppo will apply preprocessor macros on tokens returned by >> erl_scan: >> >> >>> >> >> >>> https://github.com/evanmiller/aleppo >> >> >>> >> >> >>> I think it will fit your needs. Aleppo also lets you use -ifdef and >> >> >>> the like inside of functions since it uses a proper grammar for the >> >> >>> macro syntax. >> >> >>> >> >> >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter >> >> >>> wrote: >> >> >>> > I'd like to construct a parse transform that has a macro in the >> >> >>> > inserted >> >> >>> > code. I've had no problem using erl_scan:tokens and >> >> >>> > erl_parse:parse_form to >> >> >>> > construct the AST as long as no macros exist. >> >> >>> > >> >> >>> > Since macros are expanded before getting to this point it fails >> if >> >> >>> > in >> >> >>> > the >> >> >>> > string scanned has something like ?MODULE. I looked at epp_dodger >> >> >>> > and >> >> >>> > elsewhere but I can't seem to find something I could use to send >> in >> >> >>> > a >> >> >>> > string >> >> >>> > like: >> >> >>> > >> >> >>> > "init() -> ?MODULE." >> >> >>> > >> >> >>> > And have tokens I could use to then generate the AST. >> >> >>> > >> >> >>> > Is this possible? >> >> >>> > >> >> >>> > Thanks, >> >> >>> > Tristan >> >> >>> > >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Evan Miller >> >> >>> http://www.evanmiller.org/ >> >> >> >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Evan Miller >> >> http://www.evanmiller.org/ >> > >> > >> >> >> >> -- >> Evan Miller >> http://www.evanmiller.org/ >> > > From bile@REDACTED Sun Mar 6 20:17:18 2011 From: bile@REDACTED (bile@REDACTED) Date: Sun, 6 Mar 2011 14:17:18 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> Message-ID: <20110306141718.0eb83211@firefly> Such as? There are lots of random DBs available but any with the ease of use and speed mnesia seems to offer? On Sun, 6 Mar 2011 10:01:24 -0800 Bob Ippolito wrote: > Well, I'm sure you do know what you're doing in general, but not with > Mnesia. Mnesia is quirky. You will probably eventually regret using > Mnesia. We try and avoid it as much as possible, because of its > inability to do the right thing with network partitions and the dets > limitations for disk tables. If you can afford to, I'd look at using > something else instead. > > On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: > > Yeah but to be blunt, that's two responses with "if you know what > > you're doing". ?I know what I'm doing, I just have no idea what > > Mnesia is doing. ?The documentation is inconsistent with itself and > > with Gudmundsson's remarks. ?Perhaps switch it off for any table > > spread across multiple nodes if no guarantees can be made in that > > case? > > > > > > On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito > > wrote: > >> Well, they're actually quite useful (for performance) if you know > >> what you're doing and the application can tolerate it. Often times > >> the return value of a counter is not important. I suppose if they > >> were ripped out, maybe fewer people would use Mnesia, which might > >> be a net good :) > >> > >> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis > >> wrote: > >>> Amen, the whole dirty_* set of functions should be ripped out of > >>> the release. > >>> > >>> > >>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson > >>> wrote: > >>>> Nothing is promised with dirty_write don't use it. > >>>> > >>>> Dirty read may be ok if you know what you are doing > >>>> but avoid dirty_write in multiple node system. > >>>> > >>>> /Dan > >>>> > >>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis > >>>> wrote: > >>>>> Hi Ulf, > >>>>> > >>>>> Thanks for the clarification. ?I also went back to that User > >>>>> Guide and it does an admirable job of defining atomic as > >>>>> meaning the ACID all-or-nothing variety and not what for > >>>>> example you'd call atomic when incrementing counters within the > >>>>> linux kernel. ?My bad, forgot I was dealing with a database. > >>>>> ?So I think in that sense the dirty_update_counter is atomic > >>>>> across all nodes. ?I guess the question is when the users guide > >>>>> says atomicity is lost with dirty operations, what exactly is > >>>>> non-atomic about dirty_write or any dirty operation given that > >>>>> "Mnesia also ensures that all replicas of a table are updated > >>>>> if a dirty write operation is [successfully?] performed on a > >>>>> [any?] table". ?Isn't that the definition of all-or-nothing > >>>>> atomicity? Unless that statement is inaccurate, that would seem > >>>>> to mean all dirty_* operations are atomic, and that dirty is a > >>>>> reference to what an RDBMAS would call dirty reads, with an > >>>>> additional qualification that it can break isolation of > >>>>> transactions. ?Which by the way means there is no isolation > >>>>> guarantee for any transaction, since dirty operations > >>>>> compromise their isolation. ?(IMaybe filthy_* would be a better > >>>>> prefix :). ?But I suppose for now I'll take the statement about > >>>>> lack of atomicity as true, and that dirty writes can be only > >>>>> partially executed and can leave tables on different nodes in > >>>>> an inconsistent state. > >>>>> > >>>>> Ken > >>>>> > >>>>> > >>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger > >>>>> wrote: > >>>>>> > >>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: > >>>>>> > >>>>>> After running test_counter:run(5000,1) on each node > >>>>>> simultaneously, and comparing the files, i get 2893 duplicates > >>>>>> out of 5000. ?Although the counter has been incremented by > >>>>>> 10000. ?I can't repro running on a single node with a large > >>>>>> number of processes, so it seems atomic in that case. > >>>>>> > >>>>>> It is exactly the single-node case that is atomic. The > >>>>>> documentation should spell this out. > >>>>>> The User Guide does say that you lose mnesia's atomicity and > >>>>>> isolation properties if you use dirty operations, which does > >>>>>> seem to contradict what is said about dirty_update_counter(), > >>>>>> but what the Reference Manual (*and* User Guide) should say is > >>>>>> that dirty_update_counter() is atomic only in a very limited > >>>>>> context. The User Guide does say something else that is, > >>>>>> strictly speaking, wrong: "It is not possible to have > >>>>>> transaction protected updates of counter records." > >>>>>> It is of course possible - they are only records after all. > >>>>>> You can do an update counter inside a transaction through a > >>>>>> combination of read() and write(). > >>>>>> BR, > >>>>>> Ulf W > >>>>>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. > >>>>>> http://erlang-solutions.com > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> ________________________________________________________________ > >>>>> erlang-questions (at) erlang.org mailing list. > >>>>> See http://www.erlang.org/faq.html > >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>> > >>>>> > >>>> > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >>> > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From matthew@REDACTED Sun Mar 6 21:27:06 2011 From: matthew@REDACTED (Matthew Sackman) Date: Sun, 6 Mar 2011 20:27:06 +0000 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: <20110306141718.0eb83211@firefly> References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: <20110306202706.GB18258@wellquite.org> FWIW, in Rabbit we do all updates/writes in sync_transaction blocks, but do use dirty_read, dirty_select and friends (i.e. no dirty_writes or dirty_deletes[0]). Hopefully this counts as safely getting the speed we want, safely. We are in a very read-dominated scenario though. I also concur with the pain with using mnesia in a clustered environment generally: whilst I have no problem with the idea of going for consistency and availability and not coping with partitions, nevertheless, things like dealing with upgrading schemas and such in an clustered setup where you have not all nodes up, is incredibly tricky: we have to write out files so that we know which node went down last thus is safe to start mnesia on, on recovery etc etc. It's all far more painful than we'd like, but that said, these are very hard problems to get right. And it'd be a million times worse with something SQL based. And we really need transactions currently, which cuts out a number of the popular key/value stores. Matthew [0] Having just grepped the code, I find that's not quite right, but it's certainly in a non critical place! From bob@REDACTED Sun Mar 6 21:33:52 2011 From: bob@REDACTED (Bob Ippolito) Date: Sun, 6 Mar 2011 12:33:52 -0800 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: <20110306141718.0eb83211@firefly> References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: Well that's the problem. There is nothing that does a better job of exactly what Mnesia does. However, with a few years of operational experience with Mnesia, you will get network partitions and/or you'll run into the 2GB limit (or run out of RAM) and if any of those things happen you'll be in pain. Node restarts also get painfully slow when the indexes are large. If there is another data model or database that can fit your application, you might be better off without Mnesia. On Sun, Mar 6, 2011 at 11:17 AM, wrote: > Such as? There are lots of random DBs available but any with the ease > of use and speed mnesia seems to offer? > > On Sun, 6 Mar 2011 10:01:24 -0800 > Bob Ippolito wrote: > >> Well, I'm sure you do know what you're doing in general, but not with >> Mnesia. Mnesia is quirky. You will probably eventually regret using >> Mnesia. We try and avoid it as much as possible, because of its >> inability to do the right thing with network partitions and the dets >> limitations for disk tables. If you can afford to, I'd look at using >> something else instead. >> >> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: >> > Yeah but to be blunt, that's two responses with "if you know what >> > you're doing". ?I know what I'm doing, I just have no idea what >> > Mnesia is doing. ?The documentation is inconsistent with itself and >> > with Gudmundsson's remarks. ?Perhaps switch it off for any table >> > spread across multiple nodes if no guarantees can be made in that >> > case? >> > >> > >> > On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito >> > wrote: >> >> Well, they're actually quite useful (for performance) if you know >> >> what you're doing and the application can tolerate it. Often times >> >> the return value of a counter is not important. I suppose if they >> >> were ripped out, maybe fewer people would use Mnesia, which might >> >> be a net good :) >> >> >> >> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis >> >> wrote: >> >>> Amen, the whole dirty_* set of functions should be ripped out of >> >>> the release. >> >>> >> >>> >> >>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson >> >>> wrote: >> >>>> Nothing is promised with dirty_write don't use it. >> >>>> >> >>>> Dirty read may be ok if you know what you are doing >> >>>> but avoid dirty_write in multiple node system. >> >>>> >> >>>> /Dan >> >>>> >> >>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis >> >>>> wrote: >> >>>>> Hi Ulf, >> >>>>> >> >>>>> Thanks for the clarification. ?I also went back to that User >> >>>>> Guide and it does an admirable job of defining atomic as >> >>>>> meaning the ACID all-or-nothing variety and not what for >> >>>>> example you'd call atomic when incrementing counters within the >> >>>>> linux kernel. ?My bad, forgot I was dealing with a database. >> >>>>> ?So I think in that sense the dirty_update_counter is atomic >> >>>>> across all nodes. ?I guess the question is when the users guide >> >>>>> says atomicity is lost with dirty operations, what exactly is >> >>>>> non-atomic about dirty_write or any dirty operation given that >> >>>>> "Mnesia also ensures that all replicas of a table are updated >> >>>>> if a dirty write operation is [successfully?] performed on a >> >>>>> [any?] table". ?Isn't that the definition of all-or-nothing >> >>>>> atomicity? Unless that statement is inaccurate, that would seem >> >>>>> to mean all dirty_* operations are atomic, and that dirty is a >> >>>>> reference to what an RDBMAS would call dirty reads, with an >> >>>>> additional qualification that it can break isolation of >> >>>>> transactions. ?Which by the way means there is no isolation >> >>>>> guarantee for any transaction, since dirty operations >> >>>>> compromise their isolation. ?(IMaybe filthy_* would be a better >> >>>>> prefix :). ?But I suppose for now I'll take the statement about >> >>>>> lack of atomicity as true, and that dirty writes can be only >> >>>>> partially executed and can leave tables on different nodes in >> >>>>> an inconsistent state. >> >>>>> >> >>>>> Ken >> >>>>> >> >>>>> >> >>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >> >>>>> wrote: >> >>>>>> >> >>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >> >>>>>> >> >>>>>> After running test_counter:run(5000,1) on each node >> >>>>>> simultaneously, and comparing the files, i get 2893 duplicates >> >>>>>> out of 5000. ?Although the counter has been incremented by >> >>>>>> 10000. ?I can't repro running on a single node with a large >> >>>>>> number of processes, so it seems atomic in that case. >> >>>>>> >> >>>>>> It is exactly the single-node case that is atomic. The >> >>>>>> documentation should spell this out. >> >>>>>> The User Guide does say that you lose mnesia's atomicity and >> >>>>>> isolation properties if you use dirty operations, which does >> >>>>>> seem to contradict what is said about dirty_update_counter(), >> >>>>>> but what the Reference Manual (*and* User Guide) should say is >> >>>>>> that dirty_update_counter() is atomic only in a very limited >> >>>>>> context. The User Guide does say something else that is, >> >>>>>> strictly speaking, wrong: "It is not possible to have >> >>>>>> transaction protected updates of counter records." >> >>>>>> It is of course possible - they are only records after all. >> >>>>>> You can do an update counter inside a transaction through a >> >>>>>> combination of read() and write(). >> >>>>>> BR, >> >>>>>> Ulf W >> >>>>>> Ulf Wiger,?CTO, Erlang Solutions, Ltd. >> >>>>>> http://erlang-solutions.com >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>>>> ________________________________________________________________ >> >>>>> erlang-questions (at) erlang.org mailing list. >> >>>>> See http://www.erlang.org/faq.html >> >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >>>>> >> >>>>> >> >>>> >> >>> >> >>> ________________________________________________________________ >> >>> erlang-questions (at) erlang.org mailing list. >> >>> See http://www.erlang.org/faq.html >> >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >>> >> >>> >> >> >> > >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From andrew.pennebaker@REDACTED Sun Mar 6 22:29:16 2011 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Sun, 6 Mar 2011 16:29:16 -0500 Subject: [erlang-questions] escript vs erl In-Reply-To: <8402309.2486.1298504781972.JavaMail.geo-discussion-forums@prnp16> References: <8402309.2486.1298504781972.JavaMail.geo-discussion-forums@prnp16> Message-ID: That helps with a recursive call within a module. But try this: scriptedmain.erl: -module(scriptedmain). -mode(compile). -export([meaning_of_life/0]). -import(lists, [map/2]). meaning_of_life() -> 42. main(_) -> io:format("Main: The meaning of life is ~w~n", [meaning_of_life()]). test.erl: -module(test). -import(scriptedmain, [meaning_of_life/0]). main(_) -> io:format("Test: The meaning of life is ~w~n", [meaning_of_life()]). $ escript scriptedmain.erl Main: The meaning of life is 42 $ ls *.beam ls: *.beam: No such file or directory $ escript test.erl escript: exception error: undefined function scriptedmain:meaning_of_life/0 (When -mode(compile) is added to test.erl) $ escript test.erl escript: exception error: undefined function scriptedmain:meaning_of_life/0 in function test_erl__escript__1299__446882__865649:main/1 in call from escript:run/2 in call from escript:start/1 in call from init:start_it/1 in call from init:start_em/1 Cheers, Andrew Pennebaker www.yellosoft.us On Wed, Feb 23, 2011 at 6:46 PM, Yurii Rashkovskii wrote: > Another option is to specify > > -mode(compile). > > and in your spawn/3 code use ?MODULE for module > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From tristan.sloughter@REDACTED Sun Mar 6 23:14:32 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 6 Mar 2011 16:14:32 -0600 Subject: [erlang-questions] Parse Transform with Macros In-Reply-To: References: Message-ID: Now I see from the code that there is an include option, which from what I can tell should work like : aleppo:process_tokens(Tokens, [{include, [Dirs]}]) correct? On Sun, Mar 6, 2011 at 1:44 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Is the file option so it is able to expand macros by providing a include > file as an option? > > This isn't working and neither is if they are included in the module that > I'm running the process tokens from. > > So this is only working if i just use the macro ?MACHINE which I guess > always exists... Any other and it is unable to find it: > > {function_clause, > [{lists,reverse, > [{badarg, > [{dict,fetch, > ['BASE_MODULE', > > {dict,1,16,16,8,80,48, > > {[],[],[],[],[],[],[], > > [],[],[],[],[],[],[], > [],[]}, > {{[],[],[],[],[], > [['MACHINE', > > {atom,1,'BEAM'}]], > > [],[],[],[],[],[], > [],[],[],[]}}}]}, > > {aleppo,process_tree,3}, > > > > Tristan > > > On Sun, Mar 6, 2011 at 1:24 PM, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> Sweet, that worked, thanks! >> >> Tristan >> >> >> On Sun, Mar 6, 2011 at 12:51 PM, Evan Miller wrote: >> >>> Aleppo adds an "eof" token at the end if one is not present. Perhaps >>> try removing it before passing to erl_parse? It's a new "feature" and >>> perhaps a cause of bugs. >>> >>> On Sun, Mar 6, 2011 at 12:46 PM, Tristan Sloughter >>> wrote: >>> > This was working for me with erl_scan and erl_parse. The error I'm >>> getting >>> > is: >>> > {{badmatch, >>> > {error, >>> > {0,erl_parse, >>> > ["syntax error >>> before: ", >>> > "eof"]}}}, >>> > I got the code for what I'm doing by >>> > reading: >>> http://chlorophil.blogspot.com/2007/04/erlang-macro-processor-v2-part-v.html >>> > But then I wanted to use a macro in what I am added to the module. So >>> I've >>> > modified 'ast_reversed_results' to: >>> > .... >>> > String -> >>> > {done,{ok,Tokens,LineEnd},StringRest} = >>> > erl_scan:tokens([], String, LineStart), >>> > {ok, NewTokens} = aleppo:process_tokens(Tokens), >>> > {ok, AST} = erl_parse:parse_form(NewTokens), >>> > .... >>> > But this fails with the above error. >>> > Maybe I'm doing something else wrong, I'll keep working on it. >>> > Thanks, >>> > Tristan >>> > On Sun, Mar 6, 2011 at 12:41 PM, Evan Miller >>> wrote: >>> >> >>> >> IIRC erl_parse needs a full module as input, i.e. -module() at the top >>> >> and all Erlang expressions inside of named functions. >>> >> >>> >> On Sun, Mar 6, 2011 at 11:46 AM, Tristan Sloughter >>> >> wrote: >>> >> > OK, so maybe it helps if I wrote the fun correctly :). I fixed it to >>> >> > fun() >>> >> > -> ?MACHINE end. and I get: >>> >> > {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >>> >> > >>> >> > On Sun, Mar 6, 2011 at 11:42 AM, Tristan Sloughter >>> >> > wrote: >>> >> >> >>> >> >> Hey, this is great! But the result it gives me fails when sent to >>> >> >> erl_parse: >>> >> >> > {done,{ok,Tokens2,LineEnd2},StringRest2} = erl_scan:tokens([], >>> "fun() >>> >> >> > -> >>> >> >> > ?MACHINE. ", 1). >>> >> >> {done,{ok,[{'fun',1}, >>> >> >> {'(',1}, >>> >> >> {')',1}, >>> >> >> {'->',1}, >>> >> >> {'?',1}, >>> >> >> {var,1,'MACHINE'}, >>> >> >> {dot,1}], >>> >> >> 1}, >>> >> >> []} >>> >> >> > {ok, NewTokens2} = aleppo:process_tokens(Tokens2). >>> >> >> > >>> >> >> {ok,[{'fun',1}, >>> >> >> {'(',1}, >>> >> >> {')',1}, >>> >> >> {'->',1}, >>> >> >> {atom,1,'BEAM'}, >>> >> >> {dot,1}, >>> >> >> {eof,0}]} >>> >> >> > erl_parse:parse_form(NewTokens2). >>> >> >> {error,{1,erl_parse,["syntax error before: ","'fun'"]}} >>> >> >> Any ideas? >>> >> >> Thanks. >>> >> >> On Sun, Mar 6, 2011 at 11:27 AM, Evan Miller >>> >> >> wrote: >>> >> >>> >>> >> >>> Hi Tristan, >>> >> >>> >>> >> >>> Aleppo will apply preprocessor macros on tokens returned by >>> erl_scan: >>> >> >>> >>> >> >>> https://github.com/evanmiller/aleppo >>> >> >>> >>> >> >>> I think it will fit your needs. Aleppo also lets you use -ifdef >>> and >>> >> >>> the like inside of functions since it uses a proper grammar for >>> the >>> >> >>> macro syntax. >>> >> >>> >>> >> >>> On Sun, Mar 6, 2011 at 9:41 AM, Tristan Sloughter >>> >> >>> wrote: >>> >> >>> > I'd like to construct a parse transform that has a macro in the >>> >> >>> > inserted >>> >> >>> > code. I've had no problem using erl_scan:tokens and >>> >> >>> > erl_parse:parse_form to >>> >> >>> > construct the AST as long as no macros exist. >>> >> >>> > >>> >> >>> > Since macros are expanded before getting to this point it fails >>> if >>> >> >>> > in >>> >> >>> > the >>> >> >>> > string scanned has something like ?MODULE. I looked at >>> epp_dodger >>> >> >>> > and >>> >> >>> > elsewhere but I can't seem to find something I could use to send >>> in >>> >> >>> > a >>> >> >>> > string >>> >> >>> > like: >>> >> >>> > >>> >> >>> > "init() -> ?MODULE." >>> >> >>> > >>> >> >>> > And have tokens I could use to then generate the AST. >>> >> >>> > >>> >> >>> > Is this possible? >>> >> >>> > >>> >> >>> > Thanks, >>> >> >>> > Tristan >>> >> >>> > >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> -- >>> >> >>> Evan Miller >>> >> >>> http://www.evanmiller.org/ >>> >> >> >>> >> > >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Evan Miller >>> >> http://www.evanmiller.org/ >>> > >>> > >>> >>> >>> >>> -- >>> Evan Miller >>> http://www.evanmiller.org/ >>> >> >> > From ebegumisa@REDACTED Mon Mar 7 00:38:09 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 07 Mar 2011 10:38:09 +1100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: Hi Dhananjay, I too struggled with this exact question for quite some time so I'll chime in here on the two techniques I used to solve it... On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene wrote: > While supervisors are meant to automatically restart failed processes, > there is one scenario I am as yet unable figure out which is the > idiomatic approach to implement crash recovery under the default OTP > scenarios. I have considered a solution, but being a relative newbie, > I am not sure if it is idiomatic erlang and if there are better > solutions. > > Question in short : If I have a supervisor which has a number of > dynamic children, how do I set up a mechanism where in case of a > complete system crash, all the dynamic children restart at the point > they were when the system (including the supervisor) crashed. > > Question in long : > ============= > > Sample Context : A bowling game > ------------------------------------------------- > > Lets say I am writing the software to implement the software necessary > to track various games at a bowling alley. I've set up the following > processes : > > a. Lanes : If there are 10 lanes, there are 10 processes, one for each > lane. These stay fixed for the entire duration of the program > b. Games : A group of players might get together to start a game on a > free lane. A new game will get created to track the game through its > completion. When the game is over, this process shall terminate > c. Players : Each game has a number of players. One process > "player_game" is started per player. Sample state of a player game > would include current score for the player and if the last two rolls > were strike or a spare. For the purpose of brevity, the remainder of > this mail only refers to this process and ignores the others > You could reduce complexity by having each lane process maintain it's current game (players and scores) as part of it's state. The game and player_game processes appear unnecessarily confusing to me. > Objective : > --------------- > > Assuming this is a single node implementation, if the machine were to > crash, upon machine / node restart, all the player_games should be > restarted and should be at the point where the player_games were when > the machine crashed. > > Possible supervision strategy : > -------------------------------------- > > 1. Create a simple_one_for_one supervisor player_game_sup which upon > starting up for the first time would have no children associated with > them. Use supervisor:start_child to start each process > 2. The supervisor creates an entry in a database (say mnesia) every > time it launches a new process > 3. Each player_game updates the entry every time the score gets > modified. Upon termination that entry gets deleted > 4. Post crash, the supervisor is started again (say after an > application restart or via another supervisor) > 5. (Here's the difference). By default the supervisor will not restart > the dynamically added children (all the player_games). However we > modify the init code to inspect the database and launch a player_game > for each record it finds. How? I don't think you can instruct a simple_one_for_one supervisor to create children from it's init/1 callback. From the documentation... http://www.erlang.org/doc/man/supervisor.html#Module:init-1 "...No child process is then started during the initialization phase, but all children are assumed to be started dynamically using supervisor:start_child/2..." Even if you switched to one_for_one with no child specs, I don't think you'd be able to call supervisor:start_child/2 from init/1 of the same supervisor since this function is called before the supervisor has finished initialising itself and it's the actual supervisor process doing the calling. You're likely to wait forever. AFIAK, creating dynamic children (calling supervisor:start_child/2) has to be done after the supervisor has initialised by a process other than the supervisor process. This is normally not a problem if you are calling start_child/2 during the "normal" operation of the application because the supervisor in question is likely to already be up. But here, you want to call start_child/2 at *startup*. From my experience with this precise matter, this requires some process coordination. > The player_game initialises itself to the > current state as in the database and the game(s) can continue where > it/they left off. > > My questions : > -------------------- > a. Does it make sense to move the responsibility to the supervisor to > update the database each time a new player game is started or > completed ? I personally don't see the advantage of doing this. Besides (as per my understanding of OTP design principles), a supervisor's job should be just that -- supervising workers and not doing work itself. Doing this from the your worker gen_servers make more sense to me and seems more natural. i.e Reading the scores from the DB the during player_game:init and writing them every time a score gets bumped or something similar. > b. Is it an idiomatic way to implement crash recovery There is none. It's very application specific as Jesper has indicated. I've come across a couple of wide patterns, but the details of where to put checkpoints can't be generalised. For instance; although you are specifically asking about a single node, multi-node hot take-over with no DB/persistence is another way. I was recently privy to a very interesting discussion on that technique. You might want to check it out for a future project... http://thread.gmane.org/gmane.comp.lang.erlang.general/50258/focus=50269 > c. Are there any other perhaps superior ways of implementing this? > I don't know about superior, I just don't think your first suggestion will actually work. I can offer of 2 possibilities each of which I've used... Possible supervision strategy 2a: (Loader version) -------------------------------------------------- Rather than separate dynamic children for players and games as in Strategy 1, instead, each lane stores, as part of it's state, info on the current game (the players playing on the lane and their state/scores). The supervision tree might look like this... alley_sup / \ lane_ldr ___lanes_sup_____ / | : \ lane(1) lane(2) .. lane(N) * Application has a startup configuration parameter no_of_lanes which comes from a conf file or the .app file and loaded by the alley_sup... === bowling_app.app === {application, bowling_app, [{.. {env,[{no_of_lanes,10}]}, ..}]}. === alley_sup.erl === -behaviour(supervisor). .. init([]) -> {ok, No_Of_Lanes} = application:get_env(no_of_lanes), {ok, {{one_for_one, 1, 30}, [{lanes_sup, {lanes_sup, start, []}, permanent, infinity, supervisor, [lanes_sup]}, {lanes_ldr, {lanes_ldr, start, [No_Of_Lanes]}, temporary, % Starts lanes_sup children then disappears 6000, worker, [lanes_ldr]}]}}. * lane_sup is a simple_one_for_one supervisor of any number of lanes but initially has none. * Now here is the trick: lane_ldr is a gen_server is initialised with No_Of_Lanes. It's job is to call supervisor:start_child No_Of_Lanes times at startup then vanish... === lane_ldr === -behaviour(gen_server). .. init(No_Of_Lanes) when No_Of_Lanes >= 1 -> case start_lanes(No_Of_Lanes, 0) of No_Of_Lanes -> io:format("All lanes failed to init -- quitting application.~n"), {stop, all_lanes_failed}; % Cause alley_sup to quit abnormally _ -> io:format("Lane loader exiting.~n"), ignore % One or more lanes init'ed; loader's work is done. end. start_lanes(0, E) -> E; % Return no. of lanes that have failed to init start_lanes(N, E) -> case supervisor:start_child(lanes_sup, [N]) of {ok, _} -> io:format("Started lane ~w.~n", [N]), start_lanes(N - 1, E); Err -> io:format("Error starting lane ~w: ~p.~n", [N, Err]), start_lanes(N - 1, E + 1) end. %%% These are just placeholders for compiler warnings/dialyzer handle_call(void, _, void) -> {noreply, void}. handle_cast(void, void) -> {noreply, void}. handle_info(void, void) -> {noreply, void}. terminate(_, _) -> ignore. code_change(_, void, _) -> {ok, void}. * Whenever a lane is started by the sup, it loads the most recent game from the DB, or just a simple text file (lane_1.game_state, lane_2.game_state, etc -- not a big deal if a text file gets corrupted and a game is lost so a DB might be overkill). Possibly something along the lines of... === lane.erl === -behaviour(gen_server). .. -record(player_state, {frame = 0, % NB: Removed player_name shot = 1, bonus_shot = false, last_shot = normal, prior_to_last_shot = normal, max_pins = 10, score = 0}). start(Id) -> gen_server:start_link(?MODULE, Id, []). init(Id) -> process_flag(trap_exit, true), Path = filename:join(code:priv_dir(bowling_app), "lane_" ++ integer_to_list(Id) ++ ".game_state"), % Game State is a proplist of player_state records with players' name as key % [{Player_Name1, #player_state{}}, {Player_Name2, #player_state{}}, .. ] {ok, Game_State} = try read_game_state(Path) catch _:{badmatch, {error, enoent}} -> % File not found {file:write_file(Path, "[]."), []}; _:Err -> % Discard bad state io:format("Zeroing corrupt game file ~s: ~p~n.", [Path, Err]), {file:write_file(Path, "[]."), []} end, {ok, {Game_State, Path, ..maybe some non-persisted state..}}. %% Assert the happy-case for good game state when reloading it read_game_state(Path) -> {ok, [Game_State]} = file:consult(Path), true = is_list(Game_State), lists:foreach(fun({Player_Name, Player_State}) -> true = is_list(Player_Name), true = is_record(Player_State, player_state), % Maybe do some other checks ok end, Game_State), {ok, Game_State}. .. NB: You'd probably use error_logger instead of all the io:formats. * Now whenever the score gets bumped, or a new game is starts, or a game is concluded, the lane process writes the game state to your DB, or text file. For the simple text file, you could just keep calling... write_game_state(Path, Game_State) -> ok = file:write_file(Path, io_lib:format("~p.", [Game_State])). Possible supervision strategy 2b: (Start Phase version) ------------------------------------------------------- I was tipped-off by Ulf Wiger on this thread... http://thread.gmane.org/gmane.comp.lang.erlang.general/48307/focus=48324 ... that the initailsiation/coordination done by lane_ldr in 2a above is precisely what the start phases feature of included applications is for! This requires splitting the application into two, but could be make things more manageable for larger applications. So one could get rid of lane_ldr and modify 2a to get something like... alley_sup | bowling_app | - - - - - - - -|- - - - - - - - lanes_app | | ___lanes_sup_____ / | : \ lane(1) lane(2) .. lane(N) * Split everything into two apps: the primary bowling_app and the included lanes_app. * The primary application would be pretty bare, and would start lanes_sup as if it were one of it's own modules... === bowling_app.app === {application, bowling_app, [.. {mod, {application_starter,[bowling_app,[]]}}, {included_applications, [lanes_app]}, {start_phases, [{init,[]}, {go,[]}]} .. ]}. === bowling_app.erl === -behaviour(application). .. %% Called on application:start start(normal, StartArgs) -> alley_sup:start(StartArgs). %% Called *after* entire sup tree is initialised start_phase(init, normal, []) -> % If there's a DB, initialise it here ok; start_phase(go, normal, []) -> ok. .. === alley_sup.erl === -behaviour(supervisor). .. init([]) -> {ok, {{one_for_one, 1, 30}, [{lanes_sup, {lanes_sup, start, []}, permanent, infinity, supervisor, [lanes_sup]}]}}. % Mod of included app. * Nothing else is needed in the primary app. * The second application will be responsible for spawning the dynamic children on startup... === lanes_app.app === {application, lanes_app, [.. {env,[{no_of_lanes,10}]}, {mod,{lanes,[]}}, {start_phases, [{init,[]}, {go,[]}]} .. ]}. === lanes_app.erl === -behaviour(application). .. %% NOT called start(normal, StartArgs) -> lanes_sup:start(StartArgs). %% Called *after* entire sup tree is initialised %% and corresponding bowling_app:start_phase start_phase(init, normal, []) -> ok; start_phase(go, normal, []) -> {ok, No_Of_Lanes} = application:get_env(?MODULE, no_of_lanes), true = No_Of_Lanes >= 1, case start_lanes(No_Of_Lanes, 0) of No_Of_Lanes -> io:format("All lanes failed to init -- quitting application.~n"), {error, all_lanes_failed}; % Cause app to quit abnormally _ -> ok % One or more lanes init'ed, continue. end. start_lanes(0, E) -> E; % Return no. of lanes that have failed to init start_lanes(N, E) -> case supervisor:start_child(lanes_sup, [N]) of {ok, _} -> io:format("Started lane ~w.~n", [N]), start_lanes(N - 1, E); Err -> io:format("Error starting lane ~w: ~p.~n", [N, Err]), start_lanes(N - 1, E + 1) end. === lanes_sup.erl === Same as in Strategy 2a === lane.erl === Same as in Strategy 2a Strategy 2b is cleaner to me than Strategy 2a, even though it requires splitting an application into two which many people seem to have a problem with. - Edmond - > FWIW : the code I am using to learn erlang is at > https://github.com/dnene/bowling . Its not particularly interesting at > this stage since it is still under development. > > Thanks > Dhananjay > > PS: Apologies for posting it to erlang-questions after earlier posting > it to erlang programming google group. Those monitoring the latter > will receive this question twice. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Mon Mar 7 01:17:26 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 07 Mar 2011 11:17:26 +1100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: On Sat, 05 Mar 2011 16:49:12 +1100, Dhananjay Nene wrote: > On Thu, Mar 3, 2011 at 10:29 PM, Jesper Louis Andersen > wrote: >> >> On Wed, Mar 2, 2011 at 19:02, Dhananjay Nene >> wrote: >> > Question in short : If I have a supervisor which has a number of >> > dynamic children, how do I set up a mechanism where in case of a >> > complete system crash, all the dynamic children restart at the point >> > they were when the system (including the supervisor) crashed. >> >> Your only chance is to evacuate the state somewhere else or make all >> dangerous things happen in another process. You have to decide to >> which extent you want to persist the data on disk so you can read it >> in again. Personally, I would probably be looking at persisting the >> game state. The alternative solution also includes another machine >> that can take over the games. >> >> There is a blog post of mine, >> http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-and-crashes.html >> which I'll shamelessly plug because it gives some more ideas on how to >> think about this. >> > > I read the post again, and again, and then again. While it did not > give me a straight answer, it gave me the right framework to think > through the answer, which is wonderful. > > I do have a follow up question, assuming the state is in the database > to recover from, which option of the below is considered more > idiomatic if at all there exists such a distinction > If you reconsider whether you really need a DB then... > a. Supervisor looks up state in db and feeds it to child during init Hand-wavy. > b. Child self checks state in db on init. Better. - Edmond - > > Thanks > Dhananjay > -- > ----------------------------------------------------------------------------------- > http://blog.dhananjaynene.com twitter: @dnene > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jesper.louis.andersen@REDACTED Mon Mar 7 01:25:17 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2011 01:25:17 +0100 Subject: erlang/otp build problem: snmpc_cmd.xml Message-ID: I am on: jlouis@REDACTED:~/Projects/otp$ git describe OTP_R14B01-636-gac73b2c and my 'make docs' build fails with: date=`date +"%B %e %Y"`; \ xsltproc --noout \ --stringparam outdir ../html \ --stringparam docgen "/home/jlouis/Projects/otp/lib/erl_docgen" \ --stringparam topdocdir "../../../../doc" \ --stringparam pdfdir "../pdf" \ --xinclude --stringparam mod2app_file "/home/jlouis/Projects/otp/make/x86_64-unknown-linux-gnu/mod2app.xml" \ --stringparam gendate "$date" \ --stringparam appname "snmp" \ --stringparam appver "4.19" \ -path /home/jlouis/Projects/otp/lib/erl_docgen/priv/docbuilder_dtd \ -path /home/jlouis/Projects/otp/lib/erl_docgen/priv/dtd_html_entities \ /home/jlouis/Projects/otp/lib/erl_docgen/priv/xsl/db_html.xsl book.xml warning: failed to load external entity "snmpc_cmd.xml" ref_man.xml:64: element include: XInclude error : could not load snmpc_cmd.xml, and no fallback was found make[3]: *** No rule to make target `snmpc_cmd.xml', needed by `../man1/snmpc.1'. Stop. make[3]: Leaving directory `/home/jlouis/Projects/otp/lib/snmp/doc/src' make[2]: *** [docs] Error 2 make[2]: Leaving directory `/home/jlouis/Projects/otp/lib/snmp' make[1]: *** [docs] Error 2 make[1]: Leaving directory `/home/jlouis/Projects/otp/lib' make: *** [docs] Error 2 Is there any way around this? I perhaps miss some package or have an old version of a package. -- J. From zeno490@REDACTED Mon Mar 7 03:31:38 2011 From: zeno490@REDACTED (Nicholas Frechette) Date: Sun, 6 Mar 2011 21:31:38 -0500 Subject: Efficient binary creation Message-ID: Hi, I was profiling my erlang application and found out that I am spending quite a bit of time building binaries. I was wondering what is the most efficient way to create a large binary? >From an IO list? Bin = iolist_to_binary(List). >From the front? Bin = <>. >From the back? Bin = <>. I'm currently building a list of binaries (each a couple bytes long, 17-44). Since I'm moving that data from process to process and sending it over a socket, I presume it would be best to either create an IO list of larger binaries (fewer items to copy) or a single binary and then iterate over it to send it over a socket in chuncks. Tips? Thanks! Nicholas From yujiangw@REDACTED Mon Mar 7 09:54:30 2011 From: yujiangw@REDACTED (Edward Wang) Date: Mon, 7 Mar 2011 16:54:30 +0800 Subject: Dispatch arrived gen_udp messages Message-ID: I gen_udp:open/1 a udp socket in one process, gen_udp:send/4 some packets in another using that socket. It seems process that opens the socket gets arrived udp messages. Is there a way to get these udp messages dispatched to the process send them? Of course I can dispatch them myself. I just wonder if there's build-in mechanism to do the same. -Edward From bengt.kleberg@REDACTED Mon Mar 7 10:02:10 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 7 Mar 2011 10:02:10 +0100 Subject: [erlang-questions] Dispatch arrived gen_udp messages In-Reply-To: References: Message-ID: <1299488530.5111.3.camel@seasc1137> Greetings, I think you are looking for gen_udp:controlling_process/2 Please check the documentation (ex: http://erldocs.com/R14B01/kernel/gen_udp.html?i=2&search=gen_udp#controlling_process/2. bengt On Mon, 2011-03-07 at 09:54 +0100, Edward Wang wrote: > I gen_udp:open/1 a udp socket in one process, gen_udp:send/4 some packets in > another using that socket. It seems process that opens the socket gets > arrived udp messages. Is there a way to get these udp messages dispatched to > the process send them? Of course I can dispatch them myself. I just wonder > if there's build-in mechanism to do the same. > > -Edward From jesper.louis.andersen@REDACTED Mon Mar 7 10:04:17 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2011 10:04:17 +0100 Subject: [erlang-questions] Dispatch arrived gen_udp messages In-Reply-To: References: Message-ID: Not in my experience. I use an ets table for the dispatching. I think the udp tracking code contains an example in etorrent. :) it is definitely worth generalizing eventually. J. On Mar 7, 2011 9:55 AM, "Edward Wang" wrote: > I gen_udp:open/1 a udp socket in one process, gen_udp:send/4 some packets in > another using that socket. It seems process that opens the socket gets > arrived udp messages. Is there a way to get these udp messages dispatched to > the process send them? Of course I can dispatch them myself. I just wonder > if there's build-in mechanism to do the same. > > -Edward From yujiangw@REDACTED Mon Mar 7 10:29:19 2011 From: yujiangw@REDACTED (Edward Wang) Date: Mon, 7 Mar 2011 17:29:19 +0800 Subject: [erlang-questions] Dispatch arrived gen_udp messages In-Reply-To: <1299488530.5111.3.camel@seasc1137> References: <1299488530.5111.3.camel@seasc1137> Message-ID: Thanks, Bengt. This seems to do the trick. Only that gen_udp:controlling_process/2 will have effect if being called by its current controlling process in my test (mac os x). Not sure if that'll be true for other platforms. The document doesn't explicitly say. I guess it's better to spell this out. Edward. On Mon, Mar 7, 2011 at 5:02 PM, Bengt Kleberg wrote: > Greetings, > > I think you are looking for gen_udp:controlling_process/2 > Please check the documentation (ex: > > http://erldocs.com/R14B01/kernel/gen_udp.html?i=2&search=gen_udp#controlling_process/2 > . > > > bengt > > On Mon, 2011-03-07 at 09:54 +0100, Edward Wang wrote: > > I gen_udp:open/1 a udp socket in one process, gen_udp:send/4 some packets > in > > another using that socket. It seems process that opens the socket gets > > arrived udp messages. Is there a way to get these udp messages dispatched > to > > the process send them? Of course I can dispatch them myself. I just > wonder > > if there's build-in mechanism to do the same. > > > > -Edward > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ttom.kelly@REDACTED Mon Mar 7 10:39:19 2011 From: ttom.kelly@REDACTED (tom kelly) Date: Mon, 7 Mar 2011 09:39:19 +0000 Subject: [erlang-questions] NIFs, what am I missing? In-Reply-To: References: Message-ID: Hi Matthew, Did you add the path to LD_LIBRARY_PATH? Try adding "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/mevans/atis" to your startup script. //Tom. On Fri, Mar 4, 2011 at 10:46 PM, Evans, Matthew wrote: > That wasn't it. > > I needed the -m32 option on gcc > > gcc -shared -fPIC -o idxavcspc.so idxavcspc.c -I > /thirdPartyErlang/x86_32/usr/include/ -m32 > > -----Original Message----- > From: Paul Davis [mailto:paul.joseph.davis@REDACTED] > Sent: Friday, March 04, 2011 5:11 PM > To: Evans, Matthew > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] NIFs, what am I missing? > > On Fri, Mar 4, 2011 at 4:55 PM, Evans, Matthew wrote: > > The file is there: > > 9> filelib:is_file("/home/mevans/atis/idxavcspc.so"). > > true > > > > But will not load (won't load from the module either): > > 10> erlang:load_nif("/home/mevans/atis/idxavcspc",0). > > {error,{load_failed,"Failed to load NIF library: > '/home/mevans/atis/idxavcspc.so: cannot open shared object file: No such > file or directory'"}} > > > > I'm currently confused.... > > > > > > Thanks > > > > Matt > > > > Running: R14B01 > > > > Random guess from left field, perhaps the module loader isn't > following a symlink in that path? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuncer.ayaz@REDACTED Mon Mar 7 10:53:56 2011 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 7 Mar 2011 10:53:56 +0100 Subject: [erlang-questions] erlang/otp build problem: snmpc_cmd.xml In-Reply-To: References: Message-ID: On Mon, Mar 7, 2011 at 1:25 AM, Jesper Louis Andersen wrote: > I am on: > > jlouis@REDACTED:~/Projects/otp$ git describe > OTP_R14B01-636-gac73b2c > > and my 'make docs' build fails with: > > warning: failed to load external entity "snmpc_cmd.xml" > ref_man.xml:64: element include: XInclude error : could not load > snmpc_cmd.xml, and no fallback was found > make[3]: *** No rule to make target `snmpc_cmd.xml', needed by > `../man1/snmpc.1'. ?Stop. > make[3]: Leaving directory `/home/jlouis/Projects/otp/lib/snmp/doc/src' > make[2]: *** [docs] Error 2 > make[2]: Leaving directory `/home/jlouis/Projects/otp/lib/snmp' > make[1]: *** [docs] Error 2 > make[1]: Leaving directory `/home/jlouis/Projects/otp/lib' > make: *** [docs] Error 2 > > Is there any way around this? I perhaps miss some package or have an > old version of a package. Temporary issue. You can fix 'make docs' by removing the reference to snmpc_cmd.xml in lib/snmp/doc/src/ref_man.xml, but 'make install-docs' will still fail to install man1 as it's empty. Temporary workaround: $ touch lib/snmp/SKIP $ make docs install-docs $ rm lib/snmp/SKIP From atilla.erdodi@REDACTED Mon Mar 7 12:27:39 2011 From: atilla.erdodi@REDACTED (=?ISO-8859-2?Q?Atilla_Erd=F5di?=) Date: Mon, 7 Mar 2011 12:27:39 +0100 Subject: Communicating with SpamAssassin from Erlang Message-ID: Hi everyone, I'm trying to use SpamAssassin to check the spam score of a mail from Erlang using a port. I've experimented with several options for open_port, different message formats, but haven't managed to get any reply from spamc, so it times out. Apparently it doesn't notice the end of the transmission. Hopefully the solution is something trivial, I've missed so far. (The obvious solution would be to reimplement the tcp protocol of SpamAssassin in Erlang, but I would like simpler solution for now, at least for the prototype.) The simplified code is below: SpamDServer = "127.0.0.1", Exec = os:find_executable("spamc"), Port = open_port({spawn, Exec ++ " -c -d " ++ SpamDServer}, [use_stdio]), port_command(Port, "dummy text \r\n"), port_command(Port, <<4>>), %EOT receive {Port, {data, Res}} -> io:format("Spamscore: ~p~n", [Res]); Other -> io:format("Unknown message: ~p\n", [Other]) after 10000 -> io:format("timeout\n") end, erlang:port_close(Port). Regards, Atilla From ulf.wiger@REDACTED Mon Mar 7 13:29:54 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 7 Mar 2011 12:29:54 +0000 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: A key issue here is that mnesia has basically been in maintenance mode for about a decade now. The main reason for this is that Ericsson itself is using mnesia mainly as a configuration database, and is pretty comfortable with its current capabilities. At the Erlang User Group meeting at QCon London this Thursday, I will give a short talk about what we at Erlang Solutions are doing with mnesia right now, together with some of our customers, to address the kind of problems Bob is referring to. http://www.erlang-solutions.com/etc/usergroup/london BR, Ulf On 6 Mar 2011, at 20:33, Bob Ippolito wrote: > Well that's the problem. There is nothing that does a better job of > exactly what Mnesia does. However, with a few years of operational > experience with Mnesia, you will get network partitions and/or you'll > run into the 2GB limit (or run out of RAM) and if any of those things > happen you'll be in pain. Node restarts also get painfully slow when > the indexes are large. > > If there is another data model or database that can fit your > application, you might be better off without Mnesia. > > On Sun, Mar 6, 2011 at 11:17 AM, wrote: >> Such as? There are lots of random DBs available but any with the ease >> of use and speed mnesia seems to offer? >> >> On Sun, 6 Mar 2011 10:01:24 -0800 >> Bob Ippolito wrote: >> >>> Well, I'm sure you do know what you're doing in general, but not with >>> Mnesia. Mnesia is quirky. You will probably eventually regret using >>> Mnesia. We try and avoid it as much as possible, because of its >>> inability to do the right thing with network partitions and the dets >>> limitations for disk tables. If you can afford to, I'd look at using >>> something else instead. >>> >>> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: >>>> Yeah but to be blunt, that's two responses with "if you know what >>>> you're doing". I know what I'm doing, I just have no idea what >>>> Mnesia is doing. The documentation is inconsistent with itself and >>>> with Gudmundsson's remarks. Perhaps switch it off for any table >>>> spread across multiple nodes if no guarantees can be made in that >>>> case? >>>> >>>> >>>> On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito >>>> wrote: >>>>> Well, they're actually quite useful (for performance) if you know >>>>> what you're doing and the application can tolerate it. Often times >>>>> the return value of a counter is not important. I suppose if they >>>>> were ripped out, maybe fewer people would use Mnesia, which might >>>>> be a net good :) >>>>> >>>>> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis >>>>> wrote: >>>>>> Amen, the whole dirty_* set of functions should be ripped out of >>>>>> the release. >>>>>> >>>>>> >>>>>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson >>>>>> wrote: >>>>>>> Nothing is promised with dirty_write don't use it. >>>>>>> >>>>>>> Dirty read may be ok if you know what you are doing >>>>>>> but avoid dirty_write in multiple node system. >>>>>>> >>>>>>> /Dan >>>>>>> >>>>>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis >>>>>>> wrote: >>>>>>>> Hi Ulf, >>>>>>>> >>>>>>>> Thanks for the clarification. I also went back to that User >>>>>>>> Guide and it does an admirable job of defining atomic as >>>>>>>> meaning the ACID all-or-nothing variety and not what for >>>>>>>> example you'd call atomic when incrementing counters within the >>>>>>>> linux kernel. My bad, forgot I was dealing with a database. >>>>>>>> So I think in that sense the dirty_update_counter is atomic >>>>>>>> across all nodes. I guess the question is when the users guide >>>>>>>> says atomicity is lost with dirty operations, what exactly is >>>>>>>> non-atomic about dirty_write or any dirty operation given that >>>>>>>> "Mnesia also ensures that all replicas of a table are updated >>>>>>>> if a dirty write operation is [successfully?] performed on a >>>>>>>> [any?] table". Isn't that the definition of all-or-nothing >>>>>>>> atomicity? Unless that statement is inaccurate, that would seem >>>>>>>> to mean all dirty_* operations are atomic, and that dirty is a >>>>>>>> reference to what an RDBMAS would call dirty reads, with an >>>>>>>> additional qualification that it can break isolation of >>>>>>>> transactions. Which by the way means there is no isolation >>>>>>>> guarantee for any transaction, since dirty operations >>>>>>>> compromise their isolation. (IMaybe filthy_* would be a better >>>>>>>> prefix :). But I suppose for now I'll take the statement about >>>>>>>> lack of atomicity as true, and that dirty writes can be only >>>>>>>> partially executed and can leave tables on different nodes in >>>>>>>> an inconsistent state. >>>>>>>> >>>>>>>> Ken >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>>>>>>>> >>>>>>>>> After running test_counter:run(5000,1) on each node >>>>>>>>> simultaneously, and comparing the files, i get 2893 duplicates >>>>>>>>> out of 5000. Although the counter has been incremented by >>>>>>>>> 10000. I can't repro running on a single node with a large >>>>>>>>> number of processes, so it seems atomic in that case. >>>>>>>>> >>>>>>>>> It is exactly the single-node case that is atomic. The >>>>>>>>> documentation should spell this out. >>>>>>>>> The User Guide does say that you lose mnesia's atomicity and >>>>>>>>> isolation properties if you use dirty operations, which does >>>>>>>>> seem to contradict what is said about dirty_update_counter(), >>>>>>>>> but what the Reference Manual (*and* User Guide) should say is >>>>>>>>> that dirty_update_counter() is atomic only in a very limited >>>>>>>>> context. The User Guide does say something else that is, >>>>>>>>> strictly speaking, wrong: "It is not possible to have >>>>>>>>> transaction protected updates of counter records." >>>>>>>>> It is of course possible - they are only records after all. >>>>>>>>> You can do an update counter inside a transaction through a >>>>>>>>> combination of read() and write(). >>>>>>>>> BR, >>>>>>>>> Ulf W >>>>>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>>>>>>> http://erlang-solutions.com >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> ________________________________________________________________ >>>>>>>> erlang-questions (at) erlang.org mailing list. >>>>>>>> See http://www.erlang.org/faq.html >>>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions (at) erlang.org mailing list. >>>>>> See http://www.erlang.org/faq.html >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>> >>>>>> >>>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From magnus.henoch@REDACTED Mon Mar 7 13:48:22 2011 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Mon, 7 Mar 2011 12:48:22 +0000 (GMT) Subject: [erlang-questions] Communicating with SpamAssassin from Erlang In-Reply-To: <1852889378.31791299501920097.JavaMail.root@zimbra> Message-ID: <635204260.31811299502102324.JavaMail.root@zimbra> ----- "Atilla Erd?di" wrote: > port_command(Port, <<4>>), %EOT This doesn't do what you think it does: when you hit Ctrl-D in a terminal to terminate input to some program, the terminal actually intercepts the Ctrl-D and never sends it to the program, but closes the input file descriptor. As far as I know, there is no way to close the input file descriptor of a port from Erlang, so you'll have to do it in some other way. (Write the message to a file and redirect it from there, experiment with spamc's -B option, write a port driver or a NIF, etc.) Hope this helps, Magnus From bile@REDACTED Mon Mar 7 14:12:52 2011 From: bile@REDACTED (bile@REDACTED) Date: Mon, 7 Mar 2011 08:12:52 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: <20110307081252.5433c53d@firefly> Will the talk or it's notes be available online? On Mon, 7 Mar 2011 12:29:54 +0000 Ulf Wiger wrote: > > A key issue here is that mnesia has basically been in maintenance > mode for about a decade now. The main reason for this is that > Ericsson itself is using mnesia mainly as a configuration database, > and is pretty comfortable with its current capabilities. > > At the Erlang User Group meeting at QCon London this Thursday, I will > give a short talk about what we at Erlang Solutions are doing with > mnesia right now, together with some of our customers, to address the > kind of problems Bob is referring to. > > http://www.erlang-solutions.com/etc/usergroup/london > > BR, > Ulf > > On 6 Mar 2011, at 20:33, Bob Ippolito wrote: > > > Well that's the problem. There is nothing that does a better job of > > exactly what Mnesia does. However, with a few years of operational > > experience with Mnesia, you will get network partitions and/or > > you'll run into the 2GB limit (or run out of RAM) and if any of > > those things happen you'll be in pain. Node restarts also get > > painfully slow when the indexes are large. > > > > If there is another data model or database that can fit your > > application, you might be better off without Mnesia. > > > > On Sun, Mar 6, 2011 at 11:17 AM, wrote: > >> Such as? There are lots of random DBs available but any with the > >> ease of use and speed mnesia seems to offer? > >> > >> On Sun, 6 Mar 2011 10:01:24 -0800 > >> Bob Ippolito wrote: > >> > >>> Well, I'm sure you do know what you're doing in general, but not > >>> with Mnesia. Mnesia is quirky. You will probably eventually > >>> regret using Mnesia. We try and avoid it as much as possible, > >>> because of its inability to do the right thing with network > >>> partitions and the dets limitations for disk tables. If you can > >>> afford to, I'd look at using something else instead. > >>> > >>> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis > >>> wrote: > >>>> Yeah but to be blunt, that's two responses with "if you know what > >>>> you're doing". I know what I'm doing, I just have no idea what > >>>> Mnesia is doing. The documentation is inconsistent with itself > >>>> and with Gudmundsson's remarks. Perhaps switch it off for any > >>>> table spread across multiple nodes if no guarantees can be made > >>>> in that case? > >>>> > >>>> > >>>> On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito > >>>> wrote: > >>>>> Well, they're actually quite useful (for performance) if you > >>>>> know what you're doing and the application can tolerate it. > >>>>> Often times the return value of a counter is not important. I > >>>>> suppose if they were ripped out, maybe fewer people would use > >>>>> Mnesia, which might be a net good :) > >>>>> > >>>>> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis > >>>>> wrote: > >>>>>> Amen, the whole dirty_* set of functions should be ripped out > >>>>>> of the release. > >>>>>> > >>>>>> > >>>>>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson > >>>>>> wrote: > >>>>>>> Nothing is promised with dirty_write don't use it. > >>>>>>> > >>>>>>> Dirty read may be ok if you know what you are doing > >>>>>>> but avoid dirty_write in multiple node system. > >>>>>>> > >>>>>>> /Dan > >>>>>>> > >>>>>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis > >>>>>>> wrote: > >>>>>>>> Hi Ulf, > >>>>>>>> > >>>>>>>> Thanks for the clarification. I also went back to that User > >>>>>>>> Guide and it does an admirable job of defining atomic as > >>>>>>>> meaning the ACID all-or-nothing variety and not what for > >>>>>>>> example you'd call atomic when incrementing counters within > >>>>>>>> the linux kernel. My bad, forgot I was dealing with a > >>>>>>>> database. So I think in that sense the dirty_update_counter > >>>>>>>> is atomic across all nodes. I guess the question is when > >>>>>>>> the users guide says atomicity is lost with dirty > >>>>>>>> operations, what exactly is non-atomic about dirty_write or > >>>>>>>> any dirty operation given that "Mnesia also ensures that all > >>>>>>>> replicas of a table are updated if a dirty write operation > >>>>>>>> is [successfully?] performed on a [any?] table". Isn't that > >>>>>>>> the definition of all-or-nothing atomicity? Unless that > >>>>>>>> statement is inaccurate, that would seem to mean all dirty_* > >>>>>>>> operations are atomic, and that dirty is a reference to what > >>>>>>>> an RDBMAS would call dirty reads, with an additional > >>>>>>>> qualification that it can break isolation of transactions. > >>>>>>>> Which by the way means there is no isolation guarantee for > >>>>>>>> any transaction, since dirty operations compromise their > >>>>>>>> isolation. (IMaybe filthy_* would be a better prefix :). > >>>>>>>> But I suppose for now I'll take the statement about lack of > >>>>>>>> atomicity as true, and that dirty writes can be only > >>>>>>>> partially executed and can leave tables on different nodes > >>>>>>>> in an inconsistent state. > >>>>>>>> > >>>>>>>> Ken > >>>>>>>> > >>>>>>>> > >>>>>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: > >>>>>>>>> > >>>>>>>>> After running test_counter:run(5000,1) on each node > >>>>>>>>> simultaneously, and comparing the files, i get 2893 > >>>>>>>>> duplicates out of 5000. Although the counter has been > >>>>>>>>> incremented by 10000. I can't repro running on a single > >>>>>>>>> node with a large number of processes, so it seems atomic > >>>>>>>>> in that case. > >>>>>>>>> > >>>>>>>>> It is exactly the single-node case that is atomic. The > >>>>>>>>> documentation should spell this out. > >>>>>>>>> The User Guide does say that you lose mnesia's atomicity and > >>>>>>>>> isolation properties if you use dirty operations, which does > >>>>>>>>> seem to contradict what is said about > >>>>>>>>> dirty_update_counter(), but what the Reference Manual > >>>>>>>>> (*and* User Guide) should say is that > >>>>>>>>> dirty_update_counter() is atomic only in a very limited > >>>>>>>>> context. The User Guide does say something else that is, > >>>>>>>>> strictly speaking, wrong: "It is not possible to have > >>>>>>>>> transaction protected updates of counter records." It is of > >>>>>>>>> course possible - they are only records after all. You can > >>>>>>>>> do an update counter inside a transaction through a > >>>>>>>>> combination of read() and write(). BR, > >>>>>>>>> Ulf W > >>>>>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >>>>>>>>> http://erlang-solutions.com > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> ________________________________________________________________ > >>>>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>>>> See http://www.erlang.org/faq.html > >>>>>>>> To unsubscribe; > >>>>>>>> mailto:erlang-questions-unsubscribe@REDACTED > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>> See http://www.erlang.org/faq.html > >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From chandrashekhar.mullaparthi@REDACTED Mon Mar 7 15:39:29 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 7 Mar 2011 14:39:29 +0000 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: FWIW, we use mnesia in about 20 different systems all doing various stuff in the network. Yes, mnesia doesn't handle partitioned networks very well, but we've just learned to work around this. The 2G limit is not a limit at all - just fragment your table. Our customer database of about 60 million entries is partitioned into 128 fragments and it handles about 350 million lookups a day, and the CPUs are rather bored on that machine. What it does, it does amazingly well. One can keep searching for the perfect database, or one can use what one's got. I can't wait to see what new stuff is being done with mnesia, then I can use it even more :-) cheers Chandru On 7 March 2011 12:29, Ulf Wiger wrote: > > A key issue here is that mnesia has basically been in maintenance mode for > about a decade now. The main reason for this is that Ericsson itself is > using mnesia mainly as a configuration database, and is pretty comfortable > with its current capabilities. > > At the Erlang User Group meeting at QCon London this Thursday, I will give > a short talk about what we at Erlang Solutions are doing with mnesia right > now, together with some of our customers, to address the kind of problems > Bob is referring to. > > http://www.erlang-solutions.com/etc/usergroup/london > > BR, > Ulf > > On 6 Mar 2011, at 20:33, Bob Ippolito wrote: > > > Well that's the problem. There is nothing that does a better job of > > exactly what Mnesia does. However, with a few years of operational > > experience with Mnesia, you will get network partitions and/or you'll > > run into the 2GB limit (or run out of RAM) and if any of those things > > happen you'll be in pain. Node restarts also get painfully slow when > > the indexes are large. > > > > If there is another data model or database that can fit your > > application, you might be better off without Mnesia. > > > > On Sun, Mar 6, 2011 at 11:17 AM, wrote: > >> Such as? There are lots of random DBs available but any with the ease > >> of use and speed mnesia seems to offer? > >> > >> On Sun, 6 Mar 2011 10:01:24 -0800 > >> Bob Ippolito wrote: > >> > >>> Well, I'm sure you do know what you're doing in general, but not with > >>> Mnesia. Mnesia is quirky. You will probably eventually regret using > >>> Mnesia. We try and avoid it as much as possible, because of its > >>> inability to do the right thing with network partitions and the dets > >>> limitations for disk tables. If you can afford to, I'd look at using > >>> something else instead. > >>> > >>> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: > >>>> Yeah but to be blunt, that's two responses with "if you know what > >>>> you're doing". I know what I'm doing, I just have no idea what > >>>> Mnesia is doing. The documentation is inconsistent with itself and > >>>> with Gudmundsson's remarks. Perhaps switch it off for any table > >>>> spread across multiple nodes if no guarantees can be made in that > >>>> case? > >>>> > >>>> > >>>> On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito > >>>> wrote: > >>>>> Well, they're actually quite useful (for performance) if you know > >>>>> what you're doing and the application can tolerate it. Often times > >>>>> the return value of a counter is not important. I suppose if they > >>>>> were ripped out, maybe fewer people would use Mnesia, which might > >>>>> be a net good :) > >>>>> > >>>>> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis > >>>>> wrote: > >>>>>> Amen, the whole dirty_* set of functions should be ripped out of > >>>>>> the release. > >>>>>> > >>>>>> > >>>>>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson > >>>>>> wrote: > >>>>>>> Nothing is promised with dirty_write don't use it. > >>>>>>> > >>>>>>> Dirty read may be ok if you know what you are doing > >>>>>>> but avoid dirty_write in multiple node system. > >>>>>>> > >>>>>>> /Dan > >>>>>>> > >>>>>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis > >>>>>>> wrote: > >>>>>>>> Hi Ulf, > >>>>>>>> > >>>>>>>> Thanks for the clarification. I also went back to that User > >>>>>>>> Guide and it does an admirable job of defining atomic as > >>>>>>>> meaning the ACID all-or-nothing variety and not what for > >>>>>>>> example you'd call atomic when incrementing counters within the > >>>>>>>> linux kernel. My bad, forgot I was dealing with a database. > >>>>>>>> So I think in that sense the dirty_update_counter is atomic > >>>>>>>> across all nodes. I guess the question is when the users guide > >>>>>>>> says atomicity is lost with dirty operations, what exactly is > >>>>>>>> non-atomic about dirty_write or any dirty operation given that > >>>>>>>> "Mnesia also ensures that all replicas of a table are updated > >>>>>>>> if a dirty write operation is [successfully?] performed on a > >>>>>>>> [any?] table". Isn't that the definition of all-or-nothing > >>>>>>>> atomicity? Unless that statement is inaccurate, that would seem > >>>>>>>> to mean all dirty_* operations are atomic, and that dirty is a > >>>>>>>> reference to what an RDBMAS would call dirty reads, with an > >>>>>>>> additional qualification that it can break isolation of > >>>>>>>> transactions. Which by the way means there is no isolation > >>>>>>>> guarantee for any transaction, since dirty operations > >>>>>>>> compromise their isolation. (IMaybe filthy_* would be a better > >>>>>>>> prefix :). But I suppose for now I'll take the statement about > >>>>>>>> lack of atomicity as true, and that dirty writes can be only > >>>>>>>> partially executed and can leave tables on different nodes in > >>>>>>>> an inconsistent state. > >>>>>>>> > >>>>>>>> Ken > >>>>>>>> > >>>>>>>> > >>>>>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: > >>>>>>>>> > >>>>>>>>> After running test_counter:run(5000,1) on each node > >>>>>>>>> simultaneously, and comparing the files, i get 2893 duplicates > >>>>>>>>> out of 5000. Although the counter has been incremented by > >>>>>>>>> 10000. I can't repro running on a single node with a large > >>>>>>>>> number of processes, so it seems atomic in that case. > >>>>>>>>> > >>>>>>>>> It is exactly the single-node case that is atomic. The > >>>>>>>>> documentation should spell this out. > >>>>>>>>> The User Guide does say that you lose mnesia's atomicity and > >>>>>>>>> isolation properties if you use dirty operations, which does > >>>>>>>>> seem to contradict what is said about dirty_update_counter(), > >>>>>>>>> but what the Reference Manual (*and* User Guide) should say is > >>>>>>>>> that dirty_update_counter() is atomic only in a very limited > >>>>>>>>> context. The User Guide does say something else that is, > >>>>>>>>> strictly speaking, wrong: "It is not possible to have > >>>>>>>>> transaction protected updates of counter records." > >>>>>>>>> It is of course possible - they are only records after all. > >>>>>>>>> You can do an update counter inside a transaction through a > >>>>>>>>> combination of read() and write(). > >>>>>>>>> BR, > >>>>>>>>> Ulf W > >>>>>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. > >>>>>>>>> http://erlang-solutions.com > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> ________________________________________________________________ > >>>>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>>>> See http://www.erlang.org/faq.html > >>>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>> See http://www.erlang.org/faq.html > >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >> > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ebegumisa@REDACTED Mon Mar 7 15:54:48 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 08 Mar 2011 01:54:48 +1100 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: On Tue, 08 Mar 2011 01:39:29 +1100, Chandru wrote: > FWIW, we use mnesia in about 20 different systems all doing various > stuff in > the network. Yes, mnesia doesn't handle partitioned networks very well, > but > we've just learned to work around this. The 2G limit is not a limit at > all - > just fragment your table. Our customer database of about 60 million > entries > is partitioned into 128 fragments and it handles about 350 million > lookups a > day, and the CPUs are rather bored on that machine. > At T-mobile I presume? - Edmond - > What it does, it does amazingly well. One can keep searching for the > perfect > database, or one can use what one's got. > > I can't wait to see what new stuff is being done with mnesia, then I can > use > it even more :-) > > cheers > Chandru > > On 7 March 2011 12:29, Ulf Wiger wrote: > >> >> A key issue here is that mnesia has basically been in maintenance mode >> for >> about a decade now. The main reason for this is that Ericsson itself is >> using mnesia mainly as a configuration database, and is pretty >> comfortable >> with its current capabilities. >> >> At the Erlang User Group meeting at QCon London this Thursday, I will >> give >> a short talk about what we at Erlang Solutions are doing with mnesia >> right >> now, together with some of our customers, to address the kind of >> problems >> Bob is referring to. >> >> http://www.erlang-solutions.com/etc/usergroup/london >> >> BR, >> Ulf >> >> On 6 Mar 2011, at 20:33, Bob Ippolito wrote: >> >> > Well that's the problem. There is nothing that does a better job of >> > exactly what Mnesia does. However, with a few years of operational >> > experience with Mnesia, you will get network partitions and/or you'll >> > run into the 2GB limit (or run out of RAM) and if any of those things >> > happen you'll be in pain. Node restarts also get painfully slow when >> > the indexes are large. >> > >> > If there is another data model or database that can fit your >> > application, you might be better off without Mnesia. >> > >> > On Sun, Mar 6, 2011 at 11:17 AM, wrote: >> >> Such as? There are lots of random DBs available but any with the ease >> >> of use and speed mnesia seems to offer? >> >> >> >> On Sun, 6 Mar 2011 10:01:24 -0800 >> >> Bob Ippolito wrote: >> >> >> >>> Well, I'm sure you do know what you're doing in general, but not >> with >> >>> Mnesia. Mnesia is quirky. You will probably eventually regret using >> >>> Mnesia. We try and avoid it as much as possible, because of its >> >>> inability to do the right thing with network partitions and the dets >> >>> limitations for disk tables. If you can afford to, I'd look at using >> >>> something else instead. >> >>> >> >>> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: >> >>>> Yeah but to be blunt, that's two responses with "if you know what >> >>>> you're doing". I know what I'm doing, I just have no idea what >> >>>> Mnesia is doing. The documentation is inconsistent with itself and >> >>>> with Gudmundsson's remarks. Perhaps switch it off for any table >> >>>> spread across multiple nodes if no guarantees can be made in that >> >>>> case? >> >>>> >> >>>> >> >>>> On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito >> >>>> wrote: >> >>>>> Well, they're actually quite useful (for performance) if you know >> >>>>> what you're doing and the application can tolerate it. Often times >> >>>>> the return value of a counter is not important. I suppose if they >> >>>>> were ripped out, maybe fewer people would use Mnesia, which might >> >>>>> be a net good :) >> >>>>> >> >>>>> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis >> >>>>> wrote: >> >>>>>> Amen, the whole dirty_* set of functions should be ripped out of >> >>>>>> the release. >> >>>>>> >> >>>>>> >> >>>>>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson >> >>>>>> wrote: >> >>>>>>> Nothing is promised with dirty_write don't use it. >> >>>>>>> >> >>>>>>> Dirty read may be ok if you know what you are doing >> >>>>>>> but avoid dirty_write in multiple node system. >> >>>>>>> >> >>>>>>> /Dan >> >>>>>>> >> >>>>>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis >> >>>>>>> wrote: >> >>>>>>>> Hi Ulf, >> >>>>>>>> >> >>>>>>>> Thanks for the clarification. I also went back to that User >> >>>>>>>> Guide and it does an admirable job of defining atomic as >> >>>>>>>> meaning the ACID all-or-nothing variety and not what for >> >>>>>>>> example you'd call atomic when incrementing counters within the >> >>>>>>>> linux kernel. My bad, forgot I was dealing with a database. >> >>>>>>>> So I think in that sense the dirty_update_counter is atomic >> >>>>>>>> across all nodes. I guess the question is when the users guide >> >>>>>>>> says atomicity is lost with dirty operations, what exactly is >> >>>>>>>> non-atomic about dirty_write or any dirty operation given that >> >>>>>>>> "Mnesia also ensures that all replicas of a table are updated >> >>>>>>>> if a dirty write operation is [successfully?] performed on a >> >>>>>>>> [any?] table". Isn't that the definition of all-or-nothing >> >>>>>>>> atomicity? Unless that statement is inaccurate, that would seem >> >>>>>>>> to mean all dirty_* operations are atomic, and that dirty is a >> >>>>>>>> reference to what an RDBMAS would call dirty reads, with an >> >>>>>>>> additional qualification that it can break isolation of >> >>>>>>>> transactions. Which by the way means there is no isolation >> >>>>>>>> guarantee for any transaction, since dirty operations >> >>>>>>>> compromise their isolation. (IMaybe filthy_* would be a better >> >>>>>>>> prefix :). But I suppose for now I'll take the statement about >> >>>>>>>> lack of atomicity as true, and that dirty writes can be only >> >>>>>>>> partially executed and can leave tables on different nodes in >> >>>>>>>> an inconsistent state. >> >>>>>>>> >> >>>>>>>> Ken >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >> >>>>>>>>> >> >>>>>>>>> After running test_counter:run(5000,1) on each node >> >>>>>>>>> simultaneously, and comparing the files, i get 2893 duplicates >> >>>>>>>>> out of 5000. Although the counter has been incremented by >> >>>>>>>>> 10000. I can't repro running on a single node with a large >> >>>>>>>>> number of processes, so it seems atomic in that case. >> >>>>>>>>> >> >>>>>>>>> It is exactly the single-node case that is atomic. The >> >>>>>>>>> documentation should spell this out. >> >>>>>>>>> The User Guide does say that you lose mnesia's atomicity and >> >>>>>>>>> isolation properties if you use dirty operations, which does >> >>>>>>>>> seem to contradict what is said about dirty_update_counter(), >> >>>>>>>>> but what the Reference Manual (*and* User Guide) should say is >> >>>>>>>>> that dirty_update_counter() is atomic only in a very limited >> >>>>>>>>> context. The User Guide does say something else that is, >> >>>>>>>>> strictly speaking, wrong: "It is not possible to have >> >>>>>>>>> transaction protected updates of counter records." >> >>>>>>>>> It is of course possible - they are only records after all. >> >>>>>>>>> You can do an update counter inside a transaction through a >> >>>>>>>>> combination of read() and write(). >> >>>>>>>>> BR, >> >>>>>>>>> Ulf W >> >>>>>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> >>>>>>>>> http://erlang-solutions.com >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> >> ________________________________________________________________ >> >>>>>>>> erlang-questions (at) erlang.org mailing list. >> >>>>>>>> See http://www.erlang.org/faq.html >> >>>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>>> ________________________________________________________________ >> >>>>>> erlang-questions (at) erlang.org mailing list. >> >>>>>> See http://www.erlang.org/faq.html >> >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>> >> >>> ________________________________________________________________ >> >>> erlang-questions (at) erlang.org mailing list. >> >>> See http://www.erlang.org/faq.html >> >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >>> >> >> >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> >> >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From kaellis@REDACTED Mon Mar 7 16:08:46 2011 From: kaellis@REDACTED (Ken Ellis) Date: Mon, 7 Mar 2011 10:08:46 -0500 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: Thanks Ulf, that's a very useful bit of information. A DBMS in maintenance mode for 10 years really gets me excited. We have very simple needs for Mnesia, and are using a separate data store for the eventual output of our system. But databases being what they are I don't want to rely on them being up any more than I have to, so mnesia seems to fit the bill. On Mar 7, 2011 7:31 AM, "Ulf Wiger" wrote: > > A key issue here is that mnesia has basically been in maintenance mode for about a decade now. The main reason for this is that Ericsson itself is using mnesia mainly as a configuration database, and is pretty comfortable with its current capabilities. > > At the Erlang User Group meeting at QCon London this Thursday, I will give a short talk about what we at Erlang Solutions are doing with mnesia right now, together with some of our customers, to address the kind of problems Bob is referring to. > > http://www.erlang-solutions.com/etc/usergroup/london > > BR, > Ulf > > On 6 Mar 2011, at 20:33, Bob Ippolito wrote: > >> Well that's the problem. There is nothing that does a better job of >> exactly what Mnesia does. However, with a few years of operational >> experience with Mnesia, you will get network partitions and/or you'll >> run into the 2GB limit (or run out of RAM) and if any of those things >> happen you'll be in pain. Node restarts also get painfully slow when >> the indexes are large. >> >> If there is another data model or database that can fit your >> application, you might be better off without Mnesia. >> >> On Sun, Mar 6, 2011 at 11:17 AM, wrote: >>> Such as? There are lots of random DBs available but any with the ease >>> of use and speed mnesia seems to offer? >>> >>> On Sun, 6 Mar 2011 10:01:24 -0800 >>> Bob Ippolito wrote: >>> >>>> Well, I'm sure you do know what you're doing in general, but not with >>>> Mnesia. Mnesia is quirky. You will probably eventually regret using >>>> Mnesia. We try and avoid it as much as possible, because of its >>>> inability to do the right thing with network partitions and the dets >>>> limitations for disk tables. If you can afford to, I'd look at using >>>> something else instead. >>>> >>>> On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: >>>>> Yeah but to be blunt, that's two responses with "if you know what >>>>> you're doing". I know what I'm doing, I just have no idea what >>>>> Mnesia is doing. The documentation is inconsistent with itself and >>>>> with Gudmundsson's remarks. Perhaps switch it off for any table >>>>> spread across multiple nodes if no guarantees can be made in that >>>>> case? >>>>> >>>>> >>>>> On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito >>>>> wrote: >>>>>> Well, they're actually quite useful (for performance) if you know >>>>>> what you're doing and the application can tolerate it. Often times >>>>>> the return value of a counter is not important. I suppose if they >>>>>> were ripped out, maybe fewer people would use Mnesia, which might >>>>>> be a net good :) >>>>>> >>>>>> On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis >>>>>> wrote: >>>>>>> Amen, the whole dirty_* set of functions should be ripped out of >>>>>>> the release. >>>>>>> >>>>>>> >>>>>>> On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson >>>>>>> wrote: >>>>>>>> Nothing is promised with dirty_write don't use it. >>>>>>>> >>>>>>>> Dirty read may be ok if you know what you are doing >>>>>>>> but avoid dirty_write in multiple node system. >>>>>>>> >>>>>>>> /Dan >>>>>>>> >>>>>>>> On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis >>>>>>>> wrote: >>>>>>>>> Hi Ulf, >>>>>>>>> >>>>>>>>> Thanks for the clarification. I also went back to that User >>>>>>>>> Guide and it does an admirable job of defining atomic as >>>>>>>>> meaning the ACID all-or-nothing variety and not what for >>>>>>>>> example you'd call atomic when incrementing counters within the >>>>>>>>> linux kernel. My bad, forgot I was dealing with a database. >>>>>>>>> So I think in that sense the dirty_update_counter is atomic >>>>>>>>> across all nodes. I guess the question is when the users guide >>>>>>>>> says atomicity is lost with dirty operations, what exactly is >>>>>>>>> non-atomic about dirty_write or any dirty operation given that >>>>>>>>> "Mnesia also ensures that all replicas of a table are updated >>>>>>>>> if a dirty write operation is [successfully?] performed on a >>>>>>>>> [any?] table". Isn't that the definition of all-or-nothing >>>>>>>>> atomicity? Unless that statement is inaccurate, that would seem >>>>>>>>> to mean all dirty_* operations are atomic, and that dirty is a >>>>>>>>> reference to what an RDBMAS would call dirty reads, with an >>>>>>>>> additional qualification that it can break isolation of >>>>>>>>> transactions. Which by the way means there is no isolation >>>>>>>>> guarantee for any transaction, since dirty operations >>>>>>>>> compromise their isolation. (IMaybe filthy_* would be a better >>>>>>>>> prefix :). But I suppose for now I'll take the statement about >>>>>>>>> lack of atomicity as true, and that dirty writes can be only >>>>>>>>> partially executed and can leave tables on different nodes in >>>>>>>>> an inconsistent state. >>>>>>>>> >>>>>>>>> Ken >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> On 6 Mar 2011, at 06:54, Ken Ellis wrote: >>>>>>>>>> >>>>>>>>>> After running test_counter:run(5000,1) on each node >>>>>>>>>> simultaneously, and comparing the files, i get 2893 duplicates >>>>>>>>>> out of 5000. Although the counter has been incremented by >>>>>>>>>> 10000. I can't repro running on a single node with a large >>>>>>>>>> number of processes, so it seems atomic in that case. >>>>>>>>>> >>>>>>>>>> It is exactly the single-node case that is atomic. The >>>>>>>>>> documentation should spell this out. >>>>>>>>>> The User Guide does say that you lose mnesia's atomicity and >>>>>>>>>> isolation properties if you use dirty operations, which does >>>>>>>>>> seem to contradict what is said about dirty_update_counter(), >>>>>>>>>> but what the Reference Manual (*and* User Guide) should say is >>>>>>>>>> that dirty_update_counter() is atomic only in a very limited >>>>>>>>>> context. The User Guide does say something else that is, >>>>>>>>>> strictly speaking, wrong: "It is not possible to have >>>>>>>>>> transaction protected updates of counter records." >>>>>>>>>> It is of course possible - they are only records after all. >>>>>>>>>> You can do an update counter inside a transaction through a >>>>>>>>>> combination of read() and write(). >>>>>>>>>> BR, >>>>>>>>>> Ulf W >>>>>>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>>>>>>>> http://erlang-solutions.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> ________________________________________________________________ >>>>>>>>> erlang-questions (at) erlang.org mailing list. >>>>>>>>> See http://www.erlang.org/faq.html >>>>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> ________________________________________________________________ >>>>>>> erlang-questions (at) erlang.org mailing list. >>>>>>> See http://www.erlang.org/faq.html >>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bengt.kleberg@REDACTED Mon Mar 7 16:19:49 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 7 Mar 2011 16:19:49 +0100 Subject: [erlang-questions] Dispatch arrived gen_udp messages In-Reply-To: References: <1299488530.5111.3.camel@seasc1137> Message-ID: <1299511189.5111.17.camel@seasc1137> It is likely that the caveat for gen_tcp:controlling_process/2 also holds for gen_udp: "If called by any other process than the current controlling process, {error, eperm} is returned." If so it would be nice to include it in the documentation. bengt On Mon, 2011-03-07 at 10:29 +0100, Edward Wang wrote: > Thanks, Bengt. This seems to do the trick. > > > Only that gen_udp:controlling_process/2 will have effect if being > called by its current controlling process in my test (mac os x). Not > sure if that'll be true for other platforms. The document > doesn't explicitly say. I guess it's better to spell this out. > > > Edward. > > On Mon, Mar 7, 2011 at 5:02 PM, Bengt Kleberg > wrote: > Greetings, > > I think you are looking for gen_udp:controlling_process/2 > Please check the documentation (ex: > http://erldocs.com/R14B01/kernel/gen_udp.html?i=2&search=gen_udp#controlling_process/2. > > > bengt > > > On Mon, 2011-03-07 at 09:54 +0100, Edward Wang wrote: > > I gen_udp:open/1 a udp socket in one process, gen_udp:send/4 > some packets in > > another using that socket. It seems process that opens the > socket gets > > arrived udp messages. Is there a way to get these udp > messages dispatched to > > the process send them? Of course I can dispatch them myself. > I just wonder > > if there's build-in mechanism to do the same. > > > > -Edward > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From dan.heidebrecht@REDACTED Mon Mar 7 17:01:20 2011 From: dan.heidebrecht@REDACTED (Daniel Heidebrecht) Date: Mon, 7 Mar 2011 09:01:20 -0700 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: <60899018DBCF4F5E8F5F5897725871C2@gmail.com> Hi, Any chance you will be giving this talk at the Bay Area Erlang Factory conference in May? Thanks, Dan. On Monday, March 7, 2011 at 5:29 AM, Ulf Wiger wrote: > > A key issue here is that mnesia has basically been in maintenance mode for about a decade now. The main reason for this is that Ericsson itself is using mnesia mainly as a configuration database, and is pretty comfortable with its current capabilities. > > At the Erlang User Group meeting at QCon London this Thursday, I will give a short talk about what we at Erlang Solutions are doing with mnesia right now, together with some of our customers, to address the kind of problems Bob is referring to. > > http://www.erlang-solutions.com/etc/usergroup/london > > BR, > Ulf > > On 6 Mar 2011, at 20:33, Bob Ippolito wrote: > > > Well that's the problem. There is nothing that does a better job of > > exactly what Mnesia does. However, with a few years of operational > > experience with Mnesia, you will get network partitions and/or you'll > > run into the 2GB limit (or run out of RAM) and if any of those things > > happen you'll be in pain. Node restarts also get painfully slow when > > the indexes are large. > > > > If there is another data model or database that can fit your > > application, you might be better off without Mnesia. > > > > On Sun, Mar 6, 2011 at 11:17 AM, wrote: > > > Such as? There are lots of random DBs available but any with the ease > > > of use and speed mnesia seems to offer? > > > > > > On Sun, 6 Mar 2011 10:01:24 -0800 > > > Bob Ippolito wrote: > > > > > > > Well, I'm sure you do know what you're doing in general, but not with > > > > Mnesia. Mnesia is quirky. You will probably eventually regret using > > > > Mnesia. We try and avoid it as much as possible, because of its > > > > inability to do the right thing with network partitions and the dets > > > > limitations for disk tables. If you can afford to, I'd look at using > > > > something else instead. > > > > > > > > On Sun, Mar 6, 2011 at 9:54 AM, Ken Ellis wrote: > > > > > Yeah but to be blunt, that's two responses with "if you know what > > > > > you're doing". I know what I'm doing, I just have no idea what > > > > > Mnesia is doing. The documentation is inconsistent with itself and > > > > > with Gudmundsson's remarks. Perhaps switch it off for any table > > > > > spread across multiple nodes if no guarantees can be made in that > > > > > case? > > > > > > > > > > > > > > > On Sun, Mar 6, 2011 at 12:07 PM, Bob Ippolito > > > > > wrote: > > > > > > Well, they're actually quite useful (for performance) if you know > > > > > > what you're doing and the application can tolerate it. Often times > > > > > > the return value of a counter is not important. I suppose if they > > > > > > were ripped out, maybe fewer people would use Mnesia, which might > > > > > > be a net good :) > > > > > > > > > > > > On Sun, Mar 6, 2011 at 9:00 AM, Ken Ellis > > > > > > wrote: > > > > > > > Amen, the whole dirty_* set of functions should be ripped out of > > > > > > > the release. > > > > > > > > > > > > > > > > > > > > > On Sun, Mar 6, 2011 at 11:28 AM, Dan Gudmundsson > > > > > > > wrote: > > > > > > > > Nothing is promised with dirty_write don't use it. > > > > > > > > > > > > > > > > Dirty read may be ok if you know what you are doing > > > > > > > > but avoid dirty_write in multiple node system. > > > > > > > > > > > > > > > > /Dan > > > > > > > > > > > > > > > > On Sun, Mar 6, 2011 at 5:12 PM, Ken Ellis > > > > > > > > wrote: > > > > > > > > > Hi Ulf, > > > > > > > > > > > > > > > > > > Thanks for the clarification. I also went back to that User > > > > > > > > > Guide and it does an admirable job of defining atomic as > > > > > > > > > meaning the ACID all-or-nothing variety and not what for > > > > > > > > > example you'd call atomic when incrementing counters within the > > > > > > > > > linux kernel. My bad, forgot I was dealing with a database. > > > > > > > > > So I think in that sense the dirty_update_counter is atomic > > > > > > > > > across all nodes. I guess the question is when the users guide > > > > > > > > > says atomicity is lost with dirty operations, what exactly is > > > > > > > > > non-atomic about dirty_write or any dirty operation given that > > > > > > > > > "Mnesia also ensures that all replicas of a table are updated > > > > > > > > > if a dirty write operation is [successfully?] performed on a > > > > > > > > > [any?] table". Isn't that the definition of all-or-nothing > > > > > > > > > atomicity? Unless that statement is inaccurate, that would seem > > > > > > > > > to mean all dirty_* operations are atomic, and that dirty is a > > > > > > > > > reference to what an RDBMAS would call dirty reads, with an > > > > > > > > > additional qualification that it can break isolation of > > > > > > > > > transactions. Which by the way means there is no isolation > > > > > > > > > guarantee for any transaction, since dirty operations > > > > > > > > > compromise their isolation. (IMaybe filthy_* would be a better > > > > > > > > > prefix :). But I suppose for now I'll take the statement about > > > > > > > > > lack of atomicity as true, and that dirty writes can be only > > > > > > > > > partially executed and can leave tables on different nodes in > > > > > > > > > an inconsistent state. > > > > > > > > > > > > > > > > > > Ken > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Mar 6, 2011 at 2:19 AM, Ulf Wiger > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On 6 Mar 2011, at 06:54, Ken Ellis wrote: > > > > > > > > > > > > > > > > > > > > After running test_counter:run(5000,1) on each node > > > > > > > > > > simultaneously, and comparing the files, i get 2893 duplicates > > > > > > > > > > out of 5000. Although the counter has been incremented by > > > > > > > > > > 10000. I can't repro running on a single node with a large > > > > > > > > > > number of processes, so it seems atomic in that case. > > > > > > > > > > > > > > > > > > > > It is exactly the single-node case that is atomic. The > > > > > > > > > > documentation should spell this out. > > > > > > > > > > The User Guide does say that you lose mnesia's atomicity and > > > > > > > > > > isolation properties if you use dirty operations, which does > > > > > > > > > > seem to contradict what is said about dirty_update_counter(), > > > > > > > > > > but what the Reference Manual (*and* User Guide) should say is > > > > > > > > > > that dirty_update_counter() is atomic only in a very limited > > > > > > > > > > context. The User Guide does say something else that is, > > > > > > > > > > strictly speaking, wrong: "It is not possible to have > > > > > > > > > > transaction protected updates of counter records." > > > > > > > > > > It is of course possible - they are only records after all. > > > > > > > > > > You can do an update counter inside a transaction through a > > > > > > > > > > combination of read() and write(). > > > > > > > > > > BR, > > > > > > > > > > Ulf W > > > > > > > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > > > > > > > > > http://erlang-solutions.com > > > > > > > > > > > > > > > > > > ________________________________________________________________ > > > > > > > > > erlang-questions (at) erlang.org mailing list. > > > > > > > > > See http://www.erlang.org/faq.html > > > > > > > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > > > > > > > ________________________________________________________________ > > > > > > > erlang-questions (at) erlang.org mailing list. > > > > > > > See http://www.erlang.org/faq.html > > > > > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > ________________________________________________________________ > > > > erlang-questions (at) erlang.org mailing list. > > > > See http://www.erlang.org/faq.html > > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > > ________________________________________________________________ > > > erlang-questions (at) erlang.org mailing list. > > > See http://www.erlang.org/faq.html > > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From chandrashekhar.mullaparthi@REDACTED Mon Mar 7 17:11:42 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 7 Mar 2011 16:11:42 +0000 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: On 7 March 2011 14:54, Edmond Begumisa wrote: > On Tue, 08 Mar 2011 01:39:29 +1100, Chandru < > chandrashekhar.mullaparthi@REDACTED> wrote: > > FWIW, we use mnesia in about 20 different systems all doing various stuff >> in >> the network. Yes, mnesia doesn't handle partitioned networks very well, >> but >> we've just learned to work around this. The 2G limit is not a limit at all >> - >> just fragment your table. Our customer database of about 60 million >> entries >> is partitioned into 128 fragments and it handles about 350 million lookups >> a >> day, and the CPUs are rather bored on that machine. >> >> > At T-mobile I presume? > > - Edmond - > > Yes. We are called Everything Everywhere now! T-Mobile is the brand. cheers Chandru From mlortiz@REDACTED Mon Mar 7 17:16:22 2011 From: mlortiz@REDACTED (Marcos Ortiz) Date: Mon, 07 Mar 2011 11:46:22 -0430 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> Message-ID: <1299514582.2084.2.camel@marcosluis-Aspire-5251> On Mon, 2011-03-07 at 14:39 +0000, Chandru wrote: > FWIW, we use mnesia in about 20 different systems all doing various stuff in > the network. Yes, mnesia doesn't handle partitioned networks very well, but > we've just learned to work around this. The 2G limit is not a limit at all - > just fragment your table. Our customer database of about 60 million entries > is partitioned into 128 fragments and it handles about 350 million lookups a > day, and the CPUs are rather bored on that machine. Chandru, can you share with us this knowledge in a article or a talk about this work with nmesia? Regards, -- Marcos Lu?s Ort?z Valmaseda Software Engineer Centro de Tecnolog?as de Gesti?n de Datos (DATEC) Universidad de las Ciencias Inform?ticas http://uncubanitolinuxero.blogspot.com http://www.linkedin.com/in/marcosluis2186 From mre.travel@REDACTED Mon Mar 7 22:38:33 2011 From: mre.travel@REDACTED (Mike Elliott) Date: Mon, 7 Mar 2011 13:38:33 -0800 Subject: Any functional programming people in the greater Stockholm area? Message-ID: Hi -- The document DO-178B/ED-12B specifies the means by which software can be developed for aircraft which operate in civil airspace. As such, all aircraft for which civil airworthiness certification is required and which contain software (which is pretty much all aircraft outside of purely military operations) must have that software developed in accordance to that standard. This standard was written in 1992 and, not surprisingly, has become somewhat out of date. The new version (DO-178C/ED-12C) has been in development for the last five years and seeks to address many of the shortcomings of the current document. In particular, the topic of Object-oriented Programming was addressed by forming a subgroup for Object-oriented and Related Technologies, in an effort to make the use of OO possible in this restricted, highly safety-critical environment. I am a member of that subgroup and, for the past four years, have been one of the authors of the OO supplement. Some of us, however, recognized the possibility of future use of functional programming in this problem space and did the best we could to cover the fundamental needs of functional programming without ever actually mentioning it. The subgroup consists of a dozen or so, none of whom have a background in functional programming, although as a graduate student at Edinburgh in 1980, I did get an introduction to ML with Robin Milner and Luca Cardelli. The DO-178C/ED-12C standard is being approved by an ultra-conservative group of safety professionals so we have deliberately walked very softly in this area. Nonetheless, in a recent teleconference, I mentioned that although the OO supplement had not been written with any mention of Functional Programming, we had deliberately introduced ideas such as closures and automatic garbage collection with a deliberate thought about easing the efforts of future practitioners in taking a functional approach. This brought about questions along the line of "What's Functional Programming and Why Should I Care?" which I thought would not be easily answered during the last five minutes of a two hour teleconference. We are having a plenary session (attended by probably about 100 or so) April 18-22, at the ENEA office in Kista, Sweden. I have proposed having a presentation on Functional Programming sometime during this session. But, rather than my giving the presentation, since I'm at best a novice at functional programming, I was curious as to whether there might be some local talent willing to give it a shot. This has not yet been approved for the plenary, and may not be, but I thought I'd see what the possibilities might be. Let me know -- -- ============================ Mike Elliott? ? mre.travel@REDACTED ============================ From lukas.larsson@REDACTED Mon Mar 7 23:13:10 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Mon, 7 Mar 2011 22:13:10 +0000 (GMT) Subject: [erlang-questions] escript vs erl In-Reply-To: Message-ID: <233157915.34481299535990794.JavaMail.root@zimbra> Importing only means that meaning_of_life/0 will be expanded with it's module added before it i.e. to scriptedmain:meaning_of_life/0, so you still have to compile that module. Lukas ----- Original Message ----- From: "Andrew Pennebaker" To: erlang-programming@REDACTED Cc: "Yurii Rashkovskii" , "Erlang Questions" Sent: Sunday, March 6, 2011 10:29:16 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [erlang-questions] escript vs erl That helps with a recursive call within a module. But try this: scriptedmain.erl: -module(scriptedmain). -mode(compile). -export([meaning_of_life/0]). -import(lists, [map/2]). meaning_of_life() -> 42. main(_) -> io:format("Main: The meaning of life is ~w~n", [meaning_of_life()]). test.erl: -module(test). -import(scriptedmain, [meaning_of_life/0]). main(_) -> io:format("Test: The meaning of life is ~w~n", [meaning_of_life()]). $ escript scriptedmain.erl Main: The meaning of life is 42 $ ls *.beam ls: *.beam: No such file or directory $ escript test.erl escript: exception error: undefined function scriptedmain:meaning_of_life/0 (When -mode(compile) is added to test.erl) $ escript test.erl escript: exception error: undefined function scriptedmain:meaning_of_life/0 in function test_erl__escript__1299__446882__865649:main/1 in call from escript:run/2 in call from escript:start/1 in call from init:start_it/1 in call from init:start_em/1 Cheers, Andrew Pennebaker www.yellosoft.us On Wed, Feb 23, 2011 at 6:46 PM, Yurii Rashkovskii wrote: > Another option is to specify > > -mode(compile). > > and in your spawn/3 code use ?MODULE for module > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ebegumisa@REDACTED Tue Mar 8 02:58:05 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 08 Mar 2011 12:58:05 +1100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: A third option... Strategy 2c ------------ I've found 2a and 2b useful when you want to use a simple_one_for_one sup, but need to sometimes autostart some of it's children at startup based on some persisted criteria as per your specific question. But in the case of eliminating player_game and game processes and having only lanes (which I used as an example in 2a and 2b): The lanes are always a fixed number from startup, so you could use a one_for_one lanes_sup with a child-spec list, and have that at the top-level eliminating dynamic children altogether. ____lanes_sup____ / | : \ lane(1) lane(2) ... lane(n) === lanes_sup.erl === -behaviour(supervisor). .. init([]) -> {ok, No_Of_Lanes} = application:get_env(no_of_lanes), ChildSpecs = [{Id, {lane, start, []}, permanent, 10000, worker, [lane]} || Id <- lists:seq(1,No_Of_Lanes)], {ok, {{one_for_one, 1, 30}, ChildSpecs}}. === lanes.erl === Same as 2a Now the supervisor will start the children instead of you having to do it via supervisor:start_child/2. No more need for a loader or start phases. - Edmond - On Mon, 07 Mar 2011 10:38:09 +1100, Edmond Begumisa wrote: > Hi Dhananjay, > > I too struggled with this exact question for quite some time so I'll > chime in here on the two techniques I used to solve it... > > On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene > wrote: > >> While supervisors are meant to automatically restart failed processes, >> there is one scenario I am as yet unable figure out which is the >> idiomatic approach to implement crash recovery under the default OTP >> scenarios. I have considered a solution, but being a relative newbie, >> I am not sure if it is idiomatic erlang and if there are better >> solutions. >> >> Question in short : If I have a supervisor which has a number of >> dynamic children, how do I set up a mechanism where in case of a >> complete system crash, all the dynamic children restart at the point >> they were when the system (including the supervisor) crashed. >> >> Question in long : >> ============= >> >> Sample Context : A bowling game >> ------------------------------------------------- >> >> Lets say I am writing the software to implement the software necessary >> to track various games at a bowling alley. I've set up the following >> processes : >> >> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >> lane. These stay fixed for the entire duration of the program >> b. Games : A group of players might get together to start a game on a >> free lane. A new game will get created to track the game through its >> completion. When the game is over, this process shall terminate >> c. Players : Each game has a number of players. One process >> "player_game" is started per player. Sample state of a player game >> would include current score for the player and if the last two rolls >> were strike or a spare. For the purpose of brevity, the remainder of >> this mail only refers to this process and ignores the others >> > > You could reduce complexity by having each lane process maintain it's > current game (players and scores) as part of it's state. The game and > player_game processes appear unnecessarily confusing to me. > >> Objective : >> --------------- >> >> Assuming this is a single node implementation, if the machine were to >> crash, upon machine / node restart, all the player_games should be >> restarted and should be at the point where the player_games were when >> the machine crashed. >> >> Possible supervision strategy : >> -------------------------------------- >> >> 1. Create a simple_one_for_one supervisor player_game_sup which upon >> starting up for the first time would have no children associated with >> them. Use supervisor:start_child to start each process >> 2. The supervisor creates an entry in a database (say mnesia) every >> time it launches a new process >> 3. Each player_game updates the entry every time the score gets >> modified. Upon termination that entry gets deleted >> 4. Post crash, the supervisor is started again (say after an >> application restart or via another supervisor) >> 5. (Here's the difference). By default the supervisor will not restart >> the dynamically added children (all the player_games). However we >> modify the init code to inspect the database and launch a player_game >> for each record it finds. > > How? I don't think you can instruct a simple_one_for_one supervisor to > create children from it's init/1 callback. From the documentation... > > http://www.erlang.org/doc/man/supervisor.html#Module:init-1 > > "...No child process is then started during the initialization phase, > but all children are assumed to be started dynamically using > supervisor:start_child/2..." > > Even if you switched to one_for_one with no child specs, I don't think > you'd be able to call supervisor:start_child/2 from init/1 of the same > supervisor since this function is called before the supervisor has > finished initialising itself and it's the actual supervisor process > doing the calling. You're likely to wait forever. > > AFIAK, creating dynamic children (calling supervisor:start_child/2) has > to be done after the supervisor has initialised by a process other than > the supervisor process. > > This is normally not a problem if you are calling start_child/2 during > the "normal" operation of the application because the supervisor in > question is likely to already be up. But here, you want to call > start_child/2 at *startup*. From my experience with this precise matter, > this requires some process coordination. > >> The player_game initialises itself to the >> current state as in the database and the game(s) can continue where >> it/they left off. >> >> My questions : >> -------------------- >> a. Does it make sense to move the responsibility to the supervisor to >> update the database each time a new player game is started or >> completed ? > > I personally don't see the advantage of doing this. Besides (as per my > understanding of OTP design principles), a supervisor's job should be > just that -- supervising workers and not doing work itself. > > Doing this from the your worker gen_servers make more sense to me and > seems more natural. i.e Reading the scores from the DB the during > player_game:init and writing them every time a score gets bumped or > something similar. > >> b. Is it an idiomatic way to implement crash recovery > > There is none. It's very application specific as Jesper has indicated. > > I've come across a couple of wide patterns, but the details of where to > put checkpoints can't be generalised. For instance; although you are > specifically asking about a single node, multi-node hot take-over with > no DB/persistence is another way. I was recently privy to a very > interesting discussion on that technique. You might want to check it out > for a future project... > > http://thread.gmane.org/gmane.comp.lang.erlang.general/50258/focus=50269 > >> c. Are there any other perhaps superior ways of implementing this? >> > > I don't know about superior, I just don't think your first suggestion > will actually work. I can offer of 2 possibilities each of which I've > used... > > Possible supervision strategy 2a: (Loader version) > -------------------------------------------------- > > Rather than separate dynamic children for players and games as in > Strategy 1, instead, each lane stores, as part of it's state, info on > the current game (the players playing on the lane and their > state/scores). The supervision tree might look like this... > > alley_sup > / \ > lane_ldr ___lanes_sup_____ > / | : \ > lane(1) lane(2) .. lane(N) > > * Application has a startup configuration parameter no_of_lanes which > comes from a conf file or the .app file and loaded by the alley_sup... > > === bowling_app.app === > {application, bowling_app, > [{.. > {env,[{no_of_lanes,10}]}, > ..}]}. > > === alley_sup.erl === > -behaviour(supervisor). > .. > init([]) -> > {ok, No_Of_Lanes} = application:get_env(no_of_lanes), > {ok, {{one_for_one, 1, 30}, > [{lanes_sup, > {lanes_sup, start, []}, > permanent, > infinity, > supervisor, > [lanes_sup]}, > {lanes_ldr, > {lanes_ldr, start, [No_Of_Lanes]}, > temporary, % Starts lanes_sup children then disappears > 6000, > worker, > [lanes_ldr]}]}}. > > * lane_sup is a simple_one_for_one supervisor of any number of lanes but > initially has none. > * Now here is the trick: lane_ldr is a gen_server is initialised with > No_Of_Lanes. It's job is to call supervisor:start_child No_Of_Lanes > times at startup then vanish... > > === lane_ldr === > -behaviour(gen_server). > .. > init(No_Of_Lanes) when No_Of_Lanes >= 1 -> > case start_lanes(No_Of_Lanes, 0) of > No_Of_Lanes -> > io:format("All lanes failed to init -- quitting > application.~n"), > {stop, all_lanes_failed}; % Cause alley_sup to quit > abnormally > _ -> > io:format("Lane loader exiting.~n"), > ignore % One or more lanes init'ed; loader's work is done. > end. > > start_lanes(0, E) -> > E; % Return no. of lanes that have failed to init > start_lanes(N, E) -> > case supervisor:start_child(lanes_sup, [N]) of > {ok, _} -> > io:format("Started lane ~w.~n", [N]), > start_lanes(N - 1, E); > Err -> > io:format("Error starting lane ~w: ~p.~n", [N, Err]), > start_lanes(N - 1, E + 1) > end. > > %%% These are just placeholders for compiler warnings/dialyzer > > handle_call(void, _, void) -> > {noreply, void}. > > handle_cast(void, void) -> > {noreply, void}. > > handle_info(void, void) -> > {noreply, void}. > > terminate(_, _) -> > ignore. > > code_change(_, void, _) -> > {ok, void}. > > * Whenever a lane is started by the sup, it loads the most recent game > from the DB, or just a simple text file (lane_1.game_state, > lane_2.game_state, etc -- not a big deal if a text file gets corrupted > and a game is lost so a DB might be overkill). Possibly something along > the lines of... > > === lane.erl === > -behaviour(gen_server). > .. > -record(player_state, {frame = 0, % NB: Removed player_name > shot = 1, > bonus_shot = false, > last_shot = normal, > prior_to_last_shot = normal, > max_pins = 10, > score = 0}). > > start(Id) -> > gen_server:start_link(?MODULE, Id, []). > > init(Id) -> > process_flag(trap_exit, true), > Path = filename:join(code:priv_dir(bowling_app), > "lane_" ++ integer_to_list(Id) ++ > ".game_state"), > % Game State is a proplist of player_state records with players' > name as key > % [{Player_Name1, #player_state{}}, {Player_Name2, > #player_state{}}, .. ] > {ok, Game_State} = try read_game_state(Path) > catch > _:{badmatch, {error, enoent}} -> % File not > found > {file:write_file(Path, "[]."), []}; > _:Err -> % Discard > bad state > io:format("Zeroing corrupt game file > ~s: ~p~n.", > [Path, Err]), > {file:write_file(Path, "[]."), []} > end, > {ok, {Game_State, Path, ..maybe some non-persisted state..}}. > > %% Assert the happy-case for good game state when reloading it > read_game_state(Path) -> > {ok, [Game_State]} = file:consult(Path), > true = is_list(Game_State), > lists:foreach(fun({Player_Name, Player_State}) -> > true = is_list(Player_Name), > true = is_record(Player_State, player_state), > % Maybe do some other checks > ok > end, Game_State), > {ok, Game_State}. > .. > > NB: You'd probably use error_logger instead of all the io:formats. > > * Now whenever the score gets bumped, or a new game is starts, or a game > is concluded, the lane process writes the game state to your DB, or text > file. For the simple text file, you could just keep calling... > > write_game_state(Path, Game_State) -> > ok = file:write_file(Path, io_lib:format("~p.", [Game_State])). > > Possible supervision strategy 2b: (Start Phase version) > ------------------------------------------------------- > > I was tipped-off by Ulf Wiger on this thread... > > http://thread.gmane.org/gmane.comp.lang.erlang.general/48307/focus=48324 > > ... that the initailsiation/coordination done by lane_ldr in 2a above is > precisely what the start phases feature of included applications is for! > This requires splitting the application into two, but could be make > things more manageable for larger applications. So one could get rid of > lane_ldr and modify 2a to get something like... > > alley_sup > | > bowling_app | > - - - - - - - -|- - - - - - - - > lanes_app | > | > ___lanes_sup_____ > / | : \ > lane(1) lane(2) .. lane(N) > > * Split everything into two apps: the primary bowling_app and the > included lanes_app. > * The primary application would be pretty bare, and would start > lanes_sup as if it were one of it's own modules... > > === bowling_app.app === > {application, bowling_app, > [.. > {mod, {application_starter,[bowling_app,[]]}}, > {included_applications, [lanes_app]}, > {start_phases, [{init,[]}, {go,[]}]} > .. > ]}. > > === bowling_app.erl === > -behaviour(application). > .. > %% Called on application:start > start(normal, StartArgs) -> > alley_sup:start(StartArgs). > > %% Called *after* entire sup tree is initialised > start_phase(init, normal, []) -> > % If there's a DB, initialise it here > ok; > start_phase(go, normal, []) -> > ok. > .. > > === alley_sup.erl === > -behaviour(supervisor). > .. > init([]) -> > {ok, {{one_for_one, 1, 30}, > [{lanes_sup, > {lanes_sup, start, []}, > permanent, > infinity, > supervisor, > [lanes_sup]}]}}. % Mod of included app. > > * Nothing else is needed in the primary app. > * The second application will be responsible for spawning the dynamic > children on startup... > > === lanes_app.app === > {application, lanes_app, > [.. > {env,[{no_of_lanes,10}]}, > {mod,{lanes,[]}}, > {start_phases, [{init,[]}, {go,[]}]} > .. > ]}. > > === lanes_app.erl === > -behaviour(application). > .. > %% NOT called > start(normal, StartArgs) -> > lanes_sup:start(StartArgs). > > %% Called *after* entire sup tree is initialised > %% and corresponding bowling_app:start_phase > start_phase(init, normal, []) -> > ok; > start_phase(go, normal, []) -> > {ok, No_Of_Lanes} = application:get_env(?MODULE, no_of_lanes), > true = No_Of_Lanes >= 1, > case start_lanes(No_Of_Lanes, 0) of > No_Of_Lanes -> > io:format("All lanes failed to init -- quitting > application.~n"), > {error, all_lanes_failed}; % Cause app to quit abnormally > _ -> > ok % One or more lanes init'ed, continue. > end. > > start_lanes(0, E) -> > E; % Return no. of lanes that have failed to init > start_lanes(N, E) -> > case supervisor:start_child(lanes_sup, [N]) of > {ok, _} -> > io:format("Started lane ~w.~n", [N]), > start_lanes(N - 1, E); > Err -> > io:format("Error starting lane ~w: ~p.~n", [N, Err]), > start_lanes(N - 1, E + 1) > end. > > === lanes_sup.erl === > Same as in Strategy 2a > > === lane.erl === > Same as in Strategy 2a > > Strategy 2b is cleaner to me than Strategy 2a, even though it requires > splitting an application into two which many people seem to have a > problem with. > > - Edmond - > > >> FWIW : the code I am using to learn erlang is at >> https://github.com/dnene/bowling . Its not particularly interesting at >> this stage since it is still under development. >> >> Thanks >> Dhananjay >> >> PS: Apologies for posting it to erlang-questions after earlier posting >> it to erlang programming google group. Those monitoring the latter >> will receive this question twice. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From alain.odea@REDACTED Tue Mar 8 03:11:35 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 7 Mar 2011 22:41:35 -0330 Subject: [erlang-questions] Communicating with SpamAssassin from Erlang In-Reply-To: References: Message-ID: On 2011-03-07, at 7:57, Atilla Erd?di wrote: > Hi everyone, > > I'm trying to use SpamAssassin to check the spam score of a mail from > Erlang using a port. I've experimented with several options for > open_port, different message formats, but haven't managed to get any > reply from spamc, so it times out. Apparently it doesn't notice the > end of the transmission. Hopefully the solution is something trivial, > I've missed so far. > > (The obvious solution would be to reimplement the tcp protocol of > SpamAssassin in Erlang, but I would like simpler solution for now, at > least for the prototype.) > > The simplified code is below: > > SpamDServer = "127.0.0.1", > Exec = os:find_executable("spamc"), > Port = open_port({spawn, Exec ++ " -c -d " ++ SpamDServer}, [use_stdio]), > port_command(Port, "dummy text \r\n"), > port_command(Port, <<4>>), %EOT > receive > {Port, {data, Res}} -> > io:format("Spamscore: ~p~n", [Res]); > Other -> > io:format("Unknown message: ~p\n", [Other]) > after 10000 -> > io:format("timeout\n") > end, > erlang:port_close(Port). > > > Regards, > Atilla Hi Atilla: This is probably a non-answer, but Akismet provides a REST API for spam checks and is far more accurate than SpamAssassin. Akismet is non-free, but inexpensive. Cheers, Alain From ebegumisa@REDACTED Tue Mar 8 03:16:42 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 08 Mar 2011 13:16:42 +1100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: PS: The disadvantage with 2c is that if lanes:init fails in one lane the entire application will fail to start unlike 2a and 2b that is tolerant of this. This is why I personally prefer using a loader process or start-phases. - Edmond - On Tue, 08 Mar 2011 12:58:05 +1100, Edmond Begumisa wrote: > A third option... > > Strategy 2c > ------------ > > I've found 2a and 2b useful when you want to use a simple_one_for_one > sup, but need to sometimes autostart some of it's children at startup > based on some persisted criteria as per your specific question. > > But in the case of eliminating player_game and game processes and having > only lanes (which I used as an example in 2a and 2b): The lanes are > always a fixed number from startup, so you could use a one_for_one > lanes_sup with a child-spec list, and have that at the top-level > eliminating dynamic children altogether. > > ____lanes_sup____ > / | : \ > lane(1) lane(2) ... lane(n) > > > === lanes_sup.erl === > -behaviour(supervisor). > .. > init([]) -> > {ok, No_Of_Lanes} = application:get_env(no_of_lanes), > ChildSpecs = [{Id, {lane, > start, []}, > permanent, > 10000, > worker, > [lane]} > || Id <- lists:seq(1,No_Of_Lanes)], > {ok, {{one_for_one, 1, 30}, ChildSpecs}}. > > === lanes.erl === > Same as 2a > > Now the supervisor will start the children instead of you having to do > it via supervisor:start_child/2. No more need for a loader or start > phases. > > - Edmond - > > On Mon, 07 Mar 2011 10:38:09 +1100, Edmond Begumisa > wrote: > >> Hi Dhananjay, >> >> I too struggled with this exact question for quite some time so I'll >> chime in here on the two techniques I used to solve it... >> >> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >> wrote: >> >>> While supervisors are meant to automatically restart failed processes, >>> there is one scenario I am as yet unable figure out which is the >>> idiomatic approach to implement crash recovery under the default OTP >>> scenarios. I have considered a solution, but being a relative newbie, >>> I am not sure if it is idiomatic erlang and if there are better >>> solutions. >>> >>> Question in short : If I have a supervisor which has a number of >>> dynamic children, how do I set up a mechanism where in case of a >>> complete system crash, all the dynamic children restart at the point >>> they were when the system (including the supervisor) crashed. >>> >>> Question in long : >>> ============= >>> >>> Sample Context : A bowling game >>> ------------------------------------------------- >>> >>> Lets say I am writing the software to implement the software necessary >>> to track various games at a bowling alley. I've set up the following >>> processes : >>> >>> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >>> lane. These stay fixed for the entire duration of the program >>> b. Games : A group of players might get together to start a game on a >>> free lane. A new game will get created to track the game through its >>> completion. When the game is over, this process shall terminate >>> c. Players : Each game has a number of players. One process >>> "player_game" is started per player. Sample state of a player game >>> would include current score for the player and if the last two rolls >>> were strike or a spare. For the purpose of brevity, the remainder of >>> this mail only refers to this process and ignores the others >>> >> >> You could reduce complexity by having each lane process maintain it's >> current game (players and scores) as part of it's state. The game and >> player_game processes appear unnecessarily confusing to me. >> >>> Objective : >>> --------------- >>> >>> Assuming this is a single node implementation, if the machine were to >>> crash, upon machine / node restart, all the player_games should be >>> restarted and should be at the point where the player_games were when >>> the machine crashed. >>> >>> Possible supervision strategy : >>> -------------------------------------- >>> >>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>> starting up for the first time would have no children associated with >>> them. Use supervisor:start_child to start each process >>> 2. The supervisor creates an entry in a database (say mnesia) every >>> time it launches a new process >>> 3. Each player_game updates the entry every time the score gets >>> modified. Upon termination that entry gets deleted >>> 4. Post crash, the supervisor is started again (say after an >>> application restart or via another supervisor) >>> 5. (Here's the difference). By default the supervisor will not restart >>> the dynamically added children (all the player_games). However we >>> modify the init code to inspect the database and launch a player_game >>> for each record it finds. >> >> How? I don't think you can instruct a simple_one_for_one supervisor to >> create children from it's init/1 callback. From the documentation... >> >> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >> >> "...No child process is then started during the initialization phase, >> but all children are assumed to be started dynamically using >> supervisor:start_child/2..." >> >> Even if you switched to one_for_one with no child specs, I don't think >> you'd be able to call supervisor:start_child/2 from init/1 of the same >> supervisor since this function is called before the supervisor has >> finished initialising itself and it's the actual supervisor process >> doing the calling. You're likely to wait forever. >> >> AFIAK, creating dynamic children (calling supervisor:start_child/2) has >> to be done after the supervisor has initialised by a process other than >> the supervisor process. >> >> This is normally not a problem if you are calling start_child/2 during >> the "normal" operation of the application because the supervisor in >> question is likely to already be up. But here, you want to call >> start_child/2 at *startup*. From my experience with this precise >> matter, this requires some process coordination. >> >>> The player_game initialises itself to the >>> current state as in the database and the game(s) can continue where >>> it/they left off. >>> >>> My questions : >>> -------------------- >>> a. Does it make sense to move the responsibility to the supervisor to >>> update the database each time a new player game is started or >>> completed ? >> >> I personally don't see the advantage of doing this. Besides (as per my >> understanding of OTP design principles), a supervisor's job should be >> just that -- supervising workers and not doing work itself. >> >> Doing this from the your worker gen_servers make more sense to me and >> seems more natural. i.e Reading the scores from the DB the during >> player_game:init and writing them every time a score gets bumped or >> something similar. >> >>> b. Is it an idiomatic way to implement crash recovery >> >> There is none. It's very application specific as Jesper has indicated. >> >> I've come across a couple of wide patterns, but the details of where to >> put checkpoints can't be generalised. For instance; although you are >> specifically asking about a single node, multi-node hot take-over with >> no DB/persistence is another way. I was recently privy to a very >> interesting discussion on that technique. You might want to check it >> out for a future project... >> >> http://thread.gmane.org/gmane.comp.lang.erlang.general/50258/focus=50269 >> >>> c. Are there any other perhaps superior ways of implementing this? >>> >> >> I don't know about superior, I just don't think your first suggestion >> will actually work. I can offer of 2 possibilities each of which I've >> used... >> >> Possible supervision strategy 2a: (Loader version) >> -------------------------------------------------- >> >> Rather than separate dynamic children for players and games as in >> Strategy 1, instead, each lane stores, as part of it's state, info on >> the current game (the players playing on the lane and their >> state/scores). The supervision tree might look like this... >> >> alley_sup >> / \ >> lane_ldr ___lanes_sup_____ >> / | : \ >> lane(1) lane(2) .. lane(N) >> >> * Application has a startup configuration parameter no_of_lanes which >> comes from a conf file or the .app file and loaded by the alley_sup... >> >> === bowling_app.app === >> {application, bowling_app, >> [{.. >> {env,[{no_of_lanes,10}]}, >> ..}]}. >> >> === alley_sup.erl === >> -behaviour(supervisor). >> .. >> init([]) -> >> {ok, No_Of_Lanes} = application:get_env(no_of_lanes), >> {ok, {{one_for_one, 1, 30}, >> [{lanes_sup, >> {lanes_sup, start, []}, >> permanent, >> infinity, >> supervisor, >> [lanes_sup]}, >> {lanes_ldr, >> {lanes_ldr, start, [No_Of_Lanes]}, >> temporary, % Starts lanes_sup children then disappears >> 6000, >> worker, >> [lanes_ldr]}]}}. >> >> * lane_sup is a simple_one_for_one supervisor of any number of lanes >> but initially has none. >> * Now here is the trick: lane_ldr is a gen_server is initialised with >> No_Of_Lanes. It's job is to call supervisor:start_child No_Of_Lanes >> times at startup then vanish... >> >> === lane_ldr === >> -behaviour(gen_server). >> .. >> init(No_Of_Lanes) when No_Of_Lanes >= 1 -> >> case start_lanes(No_Of_Lanes, 0) of >> No_Of_Lanes -> >> io:format("All lanes failed to init -- quitting >> application.~n"), >> {stop, all_lanes_failed}; % Cause alley_sup to quit >> abnormally >> _ -> >> io:format("Lane loader exiting.~n"), >> ignore % One or more lanes init'ed; loader's work is done. >> end. >> >> start_lanes(0, E) -> >> E; % Return no. of lanes that have failed to init >> start_lanes(N, E) -> >> case supervisor:start_child(lanes_sup, [N]) of >> {ok, _} -> >> io:format("Started lane ~w.~n", [N]), >> start_lanes(N - 1, E); >> Err -> >> io:format("Error starting lane ~w: ~p.~n", [N, Err]), >> start_lanes(N - 1, E + 1) >> end. >> >> %%% These are just placeholders for compiler warnings/dialyzer >> >> handle_call(void, _, void) -> >> {noreply, void}. >> >> handle_cast(void, void) -> >> {noreply, void}. >> >> handle_info(void, void) -> >> {noreply, void}. >> >> terminate(_, _) -> >> ignore. >> >> code_change(_, void, _) -> >> {ok, void}. >> >> * Whenever a lane is started by the sup, it loads the most recent game >> from the DB, or just a simple text file (lane_1.game_state, >> lane_2.game_state, etc -- not a big deal if a text file gets corrupted >> and a game is lost so a DB might be overkill). Possibly something along >> the lines of... >> >> === lane.erl === >> -behaviour(gen_server). >> .. >> -record(player_state, {frame = 0, % NB: Removed player_name >> shot = 1, >> bonus_shot = false, >> last_shot = normal, >> prior_to_last_shot = normal, >> max_pins = 10, >> score = 0}). >> >> start(Id) -> >> gen_server:start_link(?MODULE, Id, []). >> >> init(Id) -> >> process_flag(trap_exit, true), >> Path = filename:join(code:priv_dir(bowling_app), >> "lane_" ++ integer_to_list(Id) ++ >> ".game_state"), >> % Game State is a proplist of player_state records with players' >> name as key >> % [{Player_Name1, #player_state{}}, {Player_Name2, >> #player_state{}}, .. ] >> {ok, Game_State} = try read_game_state(Path) >> catch >> _:{badmatch, {error, enoent}} -> % File >> not found >> {file:write_file(Path, "[]."), []}; >> _:Err -> % Discard >> bad state >> io:format("Zeroing corrupt game file >> ~s: ~p~n.", >> [Path, Err]), >> {file:write_file(Path, "[]."), []} >> end, >> {ok, {Game_State, Path, ..maybe some non-persisted state..}}. >> >> %% Assert the happy-case for good game state when reloading it >> read_game_state(Path) -> >> {ok, [Game_State]} = file:consult(Path), >> true = is_list(Game_State), >> lists:foreach(fun({Player_Name, Player_State}) -> >> true = is_list(Player_Name), >> true = is_record(Player_State, player_state), >> % Maybe do some other checks >> ok >> end, Game_State), >> {ok, Game_State}. >> .. >> >> NB: You'd probably use error_logger instead of all the io:formats. >> >> * Now whenever the score gets bumped, or a new game is starts, or a >> game is concluded, the lane process writes the game state to your DB, >> or text file. For the simple text file, you could just keep calling... >> >> write_game_state(Path, Game_State) -> >> ok = file:write_file(Path, io_lib:format("~p.", [Game_State])). >> >> Possible supervision strategy 2b: (Start Phase version) >> ------------------------------------------------------- >> >> I was tipped-off by Ulf Wiger on this thread... >> >> http://thread.gmane.org/gmane.comp.lang.erlang.general/48307/focus=48324 >> >> ... that the initailsiation/coordination done by lane_ldr in 2a above >> is precisely what the start phases feature of included applications is >> for! This requires splitting the application into two, but could be >> make things more manageable for larger applications. So one could get >> rid of lane_ldr and modify 2a to get something like... >> >> alley_sup >> | >> bowling_app | >> - - - - - - - -|- - - - - - - - >> lanes_app | >> | >> ___lanes_sup_____ >> / | : \ >> lane(1) lane(2) .. lane(N) >> >> * Split everything into two apps: the primary bowling_app and the >> included lanes_app. >> * The primary application would be pretty bare, and would start >> lanes_sup as if it were one of it's own modules... >> >> === bowling_app.app === >> {application, bowling_app, >> [.. >> {mod, {application_starter,[bowling_app,[]]}}, >> {included_applications, [lanes_app]}, >> {start_phases, [{init,[]}, {go,[]}]} >> .. >> ]}. >> >> === bowling_app.erl === >> -behaviour(application). >> .. >> %% Called on application:start >> start(normal, StartArgs) -> >> alley_sup:start(StartArgs). >> >> %% Called *after* entire sup tree is initialised >> start_phase(init, normal, []) -> >> % If there's a DB, initialise it here >> ok; >> start_phase(go, normal, []) -> >> ok. >> .. >> >> === alley_sup.erl === >> -behaviour(supervisor). >> .. >> init([]) -> >> {ok, {{one_for_one, 1, 30}, >> [{lanes_sup, >> {lanes_sup, start, []}, >> permanent, >> infinity, >> supervisor, >> [lanes_sup]}]}}. % Mod of included app. >> >> * Nothing else is needed in the primary app. >> * The second application will be responsible for spawning the dynamic >> children on startup... >> >> === lanes_app.app === >> {application, lanes_app, >> [.. >> {env,[{no_of_lanes,10}]}, >> {mod,{lanes,[]}}, >> {start_phases, [{init,[]}, {go,[]}]} >> .. >> ]}. >> >> === lanes_app.erl === >> -behaviour(application). >> .. >> %% NOT called >> start(normal, StartArgs) -> >> lanes_sup:start(StartArgs). >> >> %% Called *after* entire sup tree is initialised >> %% and corresponding bowling_app:start_phase >> start_phase(init, normal, []) -> >> ok; >> start_phase(go, normal, []) -> >> {ok, No_Of_Lanes} = application:get_env(?MODULE, no_of_lanes), >> true = No_Of_Lanes >= 1, >> case start_lanes(No_Of_Lanes, 0) of >> No_Of_Lanes -> >> io:format("All lanes failed to init -- quitting >> application.~n"), >> {error, all_lanes_failed}; % Cause app to quit abnormally >> _ -> >> ok % One or more lanes init'ed, continue. >> end. >> >> start_lanes(0, E) -> >> E; % Return no. of lanes that have failed to init >> start_lanes(N, E) -> >> case supervisor:start_child(lanes_sup, [N]) of >> {ok, _} -> >> io:format("Started lane ~w.~n", [N]), >> start_lanes(N - 1, E); >> Err -> >> io:format("Error starting lane ~w: ~p.~n", [N, Err]), >> start_lanes(N - 1, E + 1) >> end. >> >> === lanes_sup.erl === >> Same as in Strategy 2a >> >> === lane.erl === >> Same as in Strategy 2a >> >> Strategy 2b is cleaner to me than Strategy 2a, even though it requires >> splitting an application into two which many people seem to have a >> problem with. >> >> - Edmond - >> >> >>> FWIW : the code I am using to learn erlang is at >>> https://github.com/dnene/bowling . Its not particularly interesting at >>> this stage since it is still under development. >>> >>> Thanks >>> Dhananjay >>> >>> PS: Apologies for posting it to erlang-questions after earlier posting >>> it to erlang programming google group. Those monitoring the latter >>> will receive this question twice. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From sysop@REDACTED Tue Mar 8 03:25:27 2011 From: sysop@REDACTED (Matt Stancliff) Date: Mon, 7 Mar 2011 18:25:27 -0800 Subject: [erlang-questions] Communicating with SpamAssassin from Erlang In-Reply-To: References: Message-ID: <09A1110A-6972-4C12-B924-A54AE1E010AC@mindspring.com> On Mar 7, 2011, at 3:27 AM, Atilla Erd?di wrote: > I'm trying to use SpamAssassin to check the spam score of a mail from > Erlang using a port. I've experimented with several options for > open_port, different message formats, but haven't managed to get any > reply from spamc, so it times out. Apparently it doesn't notice the > end of the transmission. Hopefully the solution is something trivial, > I've missed so far. You've run into a very common problem of erlang ports being unable to close stdin. > The simplified code is below: I fixed it a few weeks ago for good: https://github.com/mattsta/erlang-stdinout-pool erlang-stdinout-pool includes a port with the ability to close stdin on your external process and send stdout back to erlang. -Matt, the self proclaimed master of stdin/stdout. -- Matt Stancliff San Jose, CA @mattsta iPhone: 678-591-9337 "The best way to predict the future is to invent it." --Alan Kay From chandrashekhar.mullaparthi@REDACTED Tue Mar 8 07:39:28 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 8 Mar 2011 06:39:28 +0000 Subject: [erlang-questions] non-atomic nature of mnesia:dirty_update_counter In-Reply-To: <1299514582.2084.2.camel@marcosluis-Aspire-5251> References: <44C8A211-7F96-403A-9A79-33B7C77B74EC@erlang-solutions.com> <20110306141718.0eb83211@firefly> <1299514582.2084.2.camel@marcosluis-Aspire-5251> Message-ID: On 7 March 2011 16:16, Marcos Ortiz wrote: > On Mon, 2011-03-07 at 14:39 +0000, Chandru wrote: > > FWIW, we use mnesia in about 20 different systems all doing various stuff > in > > the network. Yes, mnesia doesn't handle partitioned networks very well, > but > > we've just learned to work around this. The 2G limit is not a limit at > all - > > just fragment your table. Our customer database of about 60 million > entries > > is partitioned into 128 fragments and it handles about 350 million > lookups a > > day, and the CPUs are rather bored on that machine. > > Chandru, can you share with us this knowledge in a article or a talk > about this work with nmesia? > > I'll try to write this up some time, but there are people such as Ulf who have far more knowledge about mnesia than I do. I suggest you monitor his work on mnesia closely :) cheers Chandru From dhananjay.nene@REDACTED Tue Mar 8 07:52:29 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Tue, 8 Mar 2011 12:22:29 +0530 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa wrote: > Hi Dhananjay, > > I too struggled with this exact question for quite some time so I'll chime > in here on the two techniques I used to solve it... > On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene > wrote: > >> >> Question in short : If I have a supervisor which has a number of >> dynamic children, how do I set up a mechanism where in case of a >> complete system crash, all the dynamic children restart at the point >> they were when the system (including the supervisor) crashed. >> >> Question in long : >> ============= >> >> Sample Context : A bowling game >> ------------------------------------------------- >> >> Lets say I am writing the software to implement the software necessary >> to track various games at a bowling alley. I've set up the following >> processes : >> >> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >> lane. These stay fixed for the entire duration of the program >> b. Games : A group of players might get together to start a game on a >> free lane. A new game will get created to track the game through its >> completion. When the game is over, this process shall terminate >> c. Players : Each game has a number of players. One process >> "player_game" is started per player. Sample state of a player game >> would include current score for the player and if the last two rolls >> were strike or a spare. For the purpose of brevity, the remainder of >> this mail only refers to this process and ignores the others >> > > You could reduce complexity by having each lane process maintain it's > current game (players and scores) as part of it's state. The game and > player_game processes appear unnecessarily confusing to me. > Interesting point. The lanes are the only static aspects of the game. I tried to consider whether it would make any difference from a client API perspective, but I imagine for a client, there is no particular reason to believe a lane is a better or worse abstraction than a game (or a player_game). >> Objective : >> --------------- >> >> Assuming this is a single node implementation, if the machine were to >> crash, upon machine / node restart, all the player_games should be >> restarted and should be at the point where the player_games were when >> the machine crashed. >> >> Possible supervision strategy : >> -------------------------------------- >> >> 1. Create a simple_one_for_one supervisor player_game_sup which upon >> starting up for the first time would have no children associated with >> them. Use supervisor:start_child to start each process >> 2. The supervisor creates an entry in a database (say mnesia) every >> time it launches a new process >> 3. Each player_game updates the entry every time the score gets >> modified. Upon termination that entry gets deleted >> 4. Post crash, the supervisor is started again (say after an >> application restart or via another supervisor) >> 5. (Here's the difference). By default the supervisor will not restart >> the dynamically added children (all the player_games). However we >> modify the init code to inspect the database and launch a player_game >> for each record it finds. > > How? I don't think you can instruct a simple_one_for_one supervisor to > create children from it's init/1 callback. From the documentation... > > http://www.erlang.org/doc/man/supervisor.html#Module:init-1 > > "...No child process is then started during the initialization phase, but > all children are assumed to be started dynamically using > supervisor:start_child/2..." Fair point. Wasn't something that struck me as an issue then, but yes, supervisor starting dynamic children inside init doesn't quite rock. > AFIAK, creating dynamic children (calling supervisor:start_child/2) has to > be done after the supervisor has initialised by a process other than the > supervisor process. Certainly. And your separate modeling of a lane_ldr (later down this mail) helps that. > This is normally not a problem if you are calling start_child/2 during the > "normal" operation of the application because the supervisor in question is > likely to already be up. But here, you want to call start_child/2 at > *startup*. From my experience with this precise matter, this requires some > process coordination. > >> The player_game initialises itself to the >> current state as in the database and the game(s) can continue where >> it/they left off. >> >> My questions : >> -------------------- >> a. Does it make sense to move the responsibility to the supervisor to >> update the database each time a new player game is started or >> completed ? > > I personally don't see the advantage of doing this. Besides (as per my > understanding of OTP design principles), a supervisor's job should be just > that -- supervising workers and not doing work itself. > > Doing this from the your worker gen_servers make more sense to me and seems > more natural. i.e Reading the scores from the DB the during player_game:init > and writing them every time a score gets bumped or something similar. > I agree > Possible supervision strategy 2a: (Loader version) > -------------------------------------------------- > > Rather than separate dynamic children for players and games as in Strategy > 1, instead, each lane stores, as part of it's state, info on the current > game (the players playing on the lane and their state/scores). The > supervision tree might look like this... > > ? ? ? ? ? alley_sup > ? ? ? ? ?/ ? ? ? ? \ > ?lane_ldr ?___lanes_sup_____ > ? ? ? ? ? / ? ? ? | ? ? : ? \ > ? ? ? ?lane(1) ?lane(2) .. lane(N) > > * Application has a startup configuration parameter no_of_lanes which comes > from a conf file or the .app file and loaded by the alley_sup... > This is a suggestion thats really had me thinking. I suspect there's a bit of the traditional OO modeling experience which is grumbling about not being able to model a game or a player game. I guess thats a matter of learning / unlearning / getting used to. > * lane_sup is a simple_one_for_one supervisor of any number of lanes but > initially has none. > * Now here is the trick: lane_ldr is a gen_server is initialised with > No_Of_Lanes. It's job is to call supervisor:start_child No_Of_Lanes times at > startup then vanish... Cool. > * Whenever a lane is started by the sup, it loads the most recent game from > the DB, or just a simple text file (lane_1.game_state, lane_2.game_state, > etc -- not a big deal if a text file gets corrupted and a game is lost so a > DB might be overkill). > * Now whenever the score gets bumped, or a new game is starts, or a game is > concluded, the lane process writes the game state to your DB, or text file. > For the simple text file, you could just keep calling... > > write_game_state(Path, Game_State) -> > ? ?ok = file:write_file(Path, io_lib:format("~p.", [Game_State])). yes, that was one the options I had in mind > Possible supervision strategy 2b: (Start Phase version) > ------------------------------------------------------- > > I was tipped-off by Ulf Wiger on this thread... > > http://thread.gmane.org/gmane.comp.lang.erlang.general/48307/focus=48324 > > ... that the initailsiation/coordination done by lane_ldr in 2a above is > precisely what the start phases feature of included applications is for! > This requires splitting the application into two, but could be make things > more manageable for larger applications. So one could get rid of lane_ldr > and modify 2a to get something like... > > ? ? ? ? ? alley_sup > ? ? ? ? ? ? ? | > ?bowling_app ?| > - - - - - - - -|- - - - - - - - > ?lanes_app ? ?| > ? ? ? ? ? ? ? | > ? ? ? ___lanes_sup_____ > ? ? ?/ ? ? ? | ? ? : ? \ > ?lane(1) ?lane(2) .. lane(N) > > * Split everything into two apps: the primary bowling_app and the included > lanes_app. > * The primary application would be pretty bare, and would start lanes_sup as > if it were one of it's own modules... Again a very interesting suggestion. Thanks. I'll certainly look into it (too hard to comment on it yet, since I'm still grokk'ing it). Once again, thanks a ton for this and the subsequent mails. They've certainly help me think more, and think much harder :) Dhananjay -- ----------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene From atilla.erdodi@REDACTED Tue Mar 8 10:39:13 2011 From: atilla.erdodi@REDACTED (=?ISO-8859-2?Q?Atilla_Erd=F5di?=) Date: Tue, 8 Mar 2011 10:39:13 +0100 Subject: [erlang-questions] Communicating with SpamAssassin from Erlang In-Reply-To: <09A1110A-6972-4C12-B924-A54AE1E010AC@mindspring.com> References: <09A1110A-6972-4C12-B924-A54AE1E010AC@mindspring.com> Message-ID: Thanks to everybody for the advices. Finally, I've implemented a subset of the spamd protocolon my own, it literally took several lines of code. Atilla 2011/3/8 Matt Stancliff : > > On Mar 7, 2011, at 3:27 AM, Atilla Erd?di wrote: > >> I'm trying to use SpamAssassin to check the spam score of a mail from >> Erlang using a port. ?I've experimented with several options for >> open_port, different message formats, but haven't managed to get any >> reply from spamc, so it times out. Apparently it doesn't notice the >> end of the transmission. Hopefully the solution is something trivial, >> I've missed so far. > > ?You've run into a very common problem of erlang ports being unable to close > stdin. > >> The simplified code is below: > > ?I fixed it a few weeks ago for good: > https://github.com/mattsta/erlang-stdinout-pool > > ?erlang-stdinout-pool includes a port with the ability to close stdin on > your external process and send stdout back to erlang. > > > -Matt, the self proclaimed master of stdin/stdout. > -- > Matt Stancliff ? ? ? ? ? ? ? ? ? ?San Jose, CA > @mattsta ? ? ? ? ? ? ? ? ?iPhone: 678-591-9337 > "The best way to predict the future is to invent it." --Alan Kay > > From ebegumisa@REDACTED Tue Mar 8 11:43:57 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 08 Mar 2011 21:43:57 +1100 Subject: [erlang-questions] (noob-help) Supervision strategies to automatically restart dynamically added children In-Reply-To: References: Message-ID: On Tue, 08 Mar 2011 17:52:29 +1100, Dhananjay Nene wrote: > On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa > wrote: >> Hi Dhananjay, >> >> I too struggled with this exact question for quite some time so I'll >> chime >> in here on the two techniques I used to solve it... >> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >> wrote: >> >>> >>> Question in short : If I have a supervisor which has a number of >>> dynamic children, how do I set up a mechanism where in case of a >>> complete system crash, all the dynamic children restart at the point >>> they were when the system (including the supervisor) crashed. >>> >>> Question in long : >>> ============= >>> >>> Sample Context : A bowling game >>> ------------------------------------------------- >>> >>> Lets say I am writing the software to implement the software necessary >>> to track various games at a bowling alley. I've set up the following >>> processes : >>> >>> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >>> lane. These stay fixed for the entire duration of the program >>> b. Games : A group of players might get together to start a game on a >>> free lane. A new game will get created to track the game through its >>> completion. When the game is over, this process shall terminate >>> c. Players : Each game has a number of players. One process >>> "player_game" is started per player. Sample state of a player game >>> would include current score for the player and if the last two rolls >>> were strike or a spare. For the purpose of brevity, the remainder of >>> this mail only refers to this process and ignores the others >>> >> >> You could reduce complexity by having each lane process maintain it's >> current game (players and scores) as part of it's state. The game and >> player_game processes appear unnecessarily confusing to me. >> > > Interesting point. The lanes are the only static aspects of the game. > I tried to consider whether it would make any difference from a client > API perspective, but I imagine for a client, there is no particular > reason to believe a lane is a better or worse abstraction than a game > (or a player_game). > >>> Objective : >>> --------------- >>> >>> Assuming this is a single node implementation, if the machine were to >>> crash, upon machine / node restart, all the player_games should be >>> restarted and should be at the point where the player_games were when >>> the machine crashed. >>> >>> Possible supervision strategy : >>> -------------------------------------- >>> >>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>> starting up for the first time would have no children associated with >>> them. Use supervisor:start_child to start each process >>> 2. The supervisor creates an entry in a database (say mnesia) every >>> time it launches a new process >>> 3. Each player_game updates the entry every time the score gets >>> modified. Upon termination that entry gets deleted >>> 4. Post crash, the supervisor is started again (say after an >>> application restart or via another supervisor) >>> 5. (Here's the difference). By default the supervisor will not restart >>> the dynamically added children (all the player_games). However we >>> modify the init code to inspect the database and launch a player_game >>> for each record it finds. >> >> How? I don't think you can instruct a simple_one_for_one supervisor to >> create children from it's init/1 callback. From the documentation... >> >> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >> >> "...No child process is then started during the initialization phase, >> but >> all children are assumed to be started dynamically using >> supervisor:start_child/2..." > > Fair point. Wasn't something that struck me as an issue then, but yes, > supervisor starting dynamic children inside init doesn't quite rock. > >> AFIAK, creating dynamic children (calling supervisor:start_child/2) has >> to >> be done after the supervisor has initialised by a process other than the >> supervisor process. > > Certainly. And your separate modeling of a lane_ldr (later down this > mail) helps that. > >> This is normally not a problem if you are calling start_child/2 during >> the >> "normal" operation of the application because the supervisor in >> question is >> likely to already be up. But here, you want to call start_child/2 at >> *startup*. From my experience with this precise matter, this requires >> some >> process coordination. >> >>> The player_game initialises itself to the >>> current state as in the database and the game(s) can continue where >>> it/they left off. >>> >>> My questions : >>> -------------------- >>> a. Does it make sense to move the responsibility to the supervisor to >>> update the database each time a new player game is started or >>> completed ? >> >> I personally don't see the advantage of doing this. Besides (as per my >> understanding of OTP design principles), a supervisor's job should be >> just >> that -- supervising workers and not doing work itself. >> >> Doing this from the your worker gen_servers make more sense to me and >> seems >> more natural. i.e Reading the scores from the DB the during >> player_game:init >> and writing them every time a score gets bumped or something similar. >> > > I agree > > >> Possible supervision strategy 2a: (Loader version) >> -------------------------------------------------- >> >> Rather than separate dynamic children for players and games as in >> Strategy >> 1, instead, each lane stores, as part of it's state, info on the current >> game (the players playing on the lane and their state/scores). The >> supervision tree might look like this... >> >> alley_sup >> / \ >> lane_ldr ___lanes_sup_____ >> / | : \ >> lane(1) lane(2) .. lane(N) >> >> * Application has a startup configuration parameter no_of_lanes which >> comes >> from a conf file or the .app file and loaded by the alley_sup... >> > > This is a suggestion thats really had me thinking. I suspect there's a > bit of the traditional OO modeling experience which is grumbling about > not being able to model a game or a player game. It's not that you can't model them, it's that you don't need to. One mantra in Erlang literature (e.g. Casarini & Thompson, pg110), is to create a process for every concurrent *activity* you observe in the real world and not every *task* you observe. So you don't necessarily need to use a process for every "object" you see in the real world. With this in mind, my immediate interpretation of your application was in two ways: A) * You have a bowling alley which has lanes. * Different _lanes_ can be *concurrently* used at the same time: map these to processes. * Only 1 player can use a _lane_ at a time: no need for player processes. * Only 1 game can take place on a _lane_ at a time: no need for game processes. * It follows that players and their game are just the state of each concurrently used _lane_. So you only need processes for lanes. Alternatively, B) * You have a bowling alley where people play games. * Several _games_ can be *concurrently* played at the same time: map these to processes. * Only 1 player can make a _game_ play at a time: no need for player processes. * Only 1 lane can be used per _game_: no need for lane processes. * It follows that players and their lane are just the state of each concurrently played game. So you only need processes for games. A) *might* be easier to implement than B) when you have to interact with hardware that manages the lane machinery, which is why I suggested it. But either way, you only need *one* class of processes. IMO, introducing more just complicates matters unnecessarily. > I guess thats a > matter of learning / unlearning / getting used to. > Modeling an your app using processes indeed very different from modeling using OO objects and takes some getting used to. It helps if you've done some multi-threaded server development before. >> * lane_sup is a simple_one_for_one supervisor of any number of lanes but >> initially has none. >> * Now here is the trick: lane_ldr is a gen_server is initialised with >> No_Of_Lanes. It's job is to call supervisor:start_child No_Of_Lanes >> times at >> startup then vanish... > > Cool. > >> * Whenever a lane is started by the sup, it loads the most recent game >> from >> the DB, or just a simple text file (lane_1.game_state, >> lane_2.game_state, >> etc -- not a big deal if a text file gets corrupted and a game is lost >> so a >> DB might be overkill). >> * Now whenever the score gets bumped, or a new game is starts, or a >> game is >> concluded, the lane process writes the game state to your DB, or text >> file. >> For the simple text file, you could just keep calling... >> >> write_game_state(Path, Game_State) -> >> ok = file:write_file(Path, io_lib:format("~p.", [Game_State])). > > yes, that was one the options I had in mind > >> Possible supervision strategy 2b: (Start Phase version) >> ------------------------------------------------------- >> >> I was tipped-off by Ulf Wiger on this thread... >> >> http://thread.gmane.org/gmane.comp.lang.erlang.general/48307/focus=48324 >> >> ... that the initailsiation/coordination done by lane_ldr in 2a above is >> precisely what the start phases feature of included applications is for! ** >> This requires splitting the application into two, ** Sorry, that statement is actually false! See below. >> but could be make things >> more manageable for larger applications. So one could get rid of >> lane_ldr >> and modify 2a to get something like... >> >> alley_sup >> | >> bowling_app | >> - - - - - - - -|- - - - - - - - >> lanes_app | >> | >> ___lanes_sup_____ >> / | : \ >> lane(1) lane(2) .. lane(N) >> >> * Split everything into two apps: the primary bowling_app and the >> included >> lanes_app. >> * The primary application would be pretty bare, and would start >> lanes_sup as >> if it were one of it's own modules... > > Again a very interesting suggestion. Thanks. I'll certainly look into > it (too hard to comment on it yet, since I'm still grokk'ing it). > Actually, I just realised that I was wrong. You don't *need* to use included applications to make use of start phases. The documentation groups start-phases under included applications so it's easy to get that impression. So you don't actually need to split the application into two as I had erroneously stated. Instead, you could simplify 2a and just have one application bowling_app with lanes_sup as it's top-level supervisor... ___lanes_sup_____ / | : \ lane(1) lane(2) .. lane(N) === bowling_app.app === {application, bowling_app, [.. {env,[{no_of_lanes,10}]}, {mod, {application_starter,[bowling,[]]}}, {start_phases, [{init,[]}, {go,[]}]} % Removed included_applications ..]}. === bowling_app.erl === -module(bowling_app). -behaviour(application). -export([start/2, stop/1, start_phase/3]). %% Called on application:start start(normal, StartArgs) -> lanes_sup:start(StartArgs). stop(_) -> ok. %% Called *after* entire sup tree is initialised start_phase(init, normal, []) -> % If there's a DB, initialise it here ok; start_phase(go, normal, []) -> {ok, No_Of_Lanes} = application:get_env(?MODULE, no_of_lanes), true = No_Of_Lanes >= 1, case start_lanes(No_Of_Lanes, 0) of No_Of_Lanes -> io:format("All lanes failed to init -- quitting application.~n"), {error, all_lanes_failed}; % Cause app to quit abnormally _ -> ok % One or more lanes init'ed, continue. end. start_lanes(0, E) -> E; % Return no. of lanes that have failed to init start_lanes(N, E) -> case supervisor:start_child(lanes_sup, [N]) of {ok, _} -> io:format("Started lane ~w.~n", [N]), start_lanes(N - 1, E); Err -> io:format("Error starting lane ~w: ~p.~n", [N, Err]), start_lanes(N - 1, E + 1) end. == lanes_sup.erl == -behaviour(supervisor). .. start(StartArgs) -> supervisor:start_link({local, ?MODULE}, ?MODULE, StartArgs). init([]) -> {ok, {{simple_one_for_one, 1, 30}, [{lane, {lane, start, []}, permanent, 10000, worker, [lane]}]}}. == lane.erl == Same as as before There. That's better! - Edmond - > Once again, thanks a ton for this and the subsequent mails. They've > certainly help me think more, and think much harder :) > > Dhananjay -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From nick.sfx.1@REDACTED Tue Mar 8 11:50:06 2011 From: nick.sfx.1@REDACTED (Nick S) Date: Tue, 8 Mar 2011 12:50:06 +0200 Subject: otp/supervisor immediately goes to "shutdown" Message-ID: Hi, Just hoping to get some help on my query http://forum.trapexit.org/viewtopic.php?t=19641 cheers, Nick From jesper.louis.andersen@REDACTED Tue Mar 8 12:23:41 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 8 Mar 2011 12:23:41 +0100 Subject: [erlang-questions] otp/supervisor immediately goes to "shutdown" In-Reply-To: References: Message-ID: On Tue, Mar 8, 2011 at 11:50, Nick S wrote: > Hi, > > Just hoping to get some help on my query > > http://forum.trapexit.org/viewtopic.php?t=19641 Your problem is probably that your start_link/1 code in the db module is written as: start_link([S]) -> where you should have written start_link(S) -> as the list is "unpacked" in the supervisor (probably due to an apply/3 form). The way you can see this is to up your debugging information on the system. Eventually boot with SASL enabled, and also look at the spawn options, notably the debug tracing (see gen_server and sys module descriptions for enabling this). My guess is though that in this case, SASL is enough as you will probably be hitting a function_clause error when the supervisor tries to call (the equivalent of apply(db, start_link, [S]). -- J. From ericbmerritt@REDACTED Tue Mar 8 15:41:23 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 Mar 2011 08:41:23 -0600 Subject: Accessing 'spec' information at runtime Message-ID: Hello All, I have been doing some digging and it I haven't found anything that indicates that spec information survives compilation. It may in the beam files, but there isn't any documentation to support that assumption in beam_lib. So before I go digging in the source I thought I would ping the list and see if anyone has any ideas on accessing 'speced' type information at run time. Thanks, Eric From kenneth.lundin@REDACTED Tue Mar 8 16:48:21 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 8 Mar 2011 16:48:21 +0100 Subject: [erlang-questions] Accessing 'spec' information at runtime In-Reply-To: References: Message-ID: The -spec and -type is included in the abstract_code chunk in the beam files. If they are compiled to contain debug_info (erlc +debug_info my_module.erl). The appear as attributes and the can be abstracted like this (only the -spec(s) in this example): %% Beam is a string contaning the path to a beam file i.e "/home/user/mymodule.beam" %% you can for example call it like this from the shell: %% m:getspecs(code:which(lists)). getspecs(Beam)-> {ok,{_,[{abstract_code,{_,AC}}]}} = beam_lib:chunks(Beam,[abstract_code]), Specs = [Spec|| Spec = {attribute,_,'spec',{FA,_}} <- AC], io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(Specs))]). Hmm, just thinking. Maybe it would be a good idea to have the -spec info available as a separate chunk who is always present even without debug_info. The -spec info could for example be used for a smarter tab-completion of functions in the shell. Including argument names and types. /Kenneth, Erlang/OTP Ericsson On Tue, Mar 8, 2011 at 3:41 PM, Eric Merritt wrote: > Hello All, > > > I have been doing some digging and it I haven't found anything that > indicates that spec information survives compilation. It may in the > beam files, but there isn't any documentation to support that > assumption in beam_lib. So before I go digging in the source I thought > I would ping the list and see if anyone has any ideas on accessing > 'speced' type information at run time. > > Thanks, > Eric > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From yrashk@REDACTED Tue Mar 8 16:46:58 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Tue, 8 Mar 2011 07:46:58 -0800 (PST) Subject: [erlang-questions] Accessing 'spec' information at runtime In-Reply-To: Message-ID: <28948482.384.1299599218447.JavaMail.geo-discussion-forums@prfx21> Compile with debug_info and use beam_lib then, you'll have attributes like: }attribute,6,type,{agner_repo,{type,6,string,[]},[]}}, and {attribute,115,spec, {{defaults,1}, [{type,115,'fun', [{type,115,product,[{type,115,agner_package_name,[]}]}, {type,115,agner_spec,[]}]}]}} -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Tue Mar 8 16:46:58 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Tue, 8 Mar 2011 07:46:58 -0800 (PST) Subject: [erlang-questions] Accessing 'spec' information at runtime In-Reply-To: Message-ID: <28948482.384.1299599218447.JavaMail.geo-discussion-forums@prfx21> Compile with debug_info and use beam_lib then, you'll have attributes like: }attribute,6,type,{agner_repo,{type,6,string,[]},[]}}, and {attribute,115,spec, {{defaults,1}, [{type,115,'fun', [{type,115,product,[{type,115,agner_package_name,[]}]}, {type,115,agner_spec,[]}]}]}} -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Tue Mar 8 17:00:29 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 Mar 2011 10:00:29 -0600 Subject: [erlang-questions] Accessing 'spec' information at runtime In-Reply-To: References: Message-ID: I think that is an extraordinarily good idea Kenneth. Having that information be a first class chunk would be useful. How do record type specs appear. I suspect they are part of the abstract code for the record. On Tue, Mar 8, 2011 at 9:48 AM, Kenneth Lundin wrote: > The -spec and -type is included in the abstract_code chunk in the beam > files. If > they are compiled to contain debug_info (erlc +debug_info my_module.erl). > > The appear as attributes and the can be abstracted like this (only the > -spec(s) in this example): > > %% Beam is a string contaning the path to a beam file i.e > "/home/user/mymodule.beam" > %% you can for example call it like this from the shell: > %% m:getspecs(code:which(lists)). > > getspecs(Beam)-> > ? ? ? ?{ok,{_,[{abstract_code,{_,AC}}]}} = > beam_lib:chunks(Beam,[abstract_code]), > ? ?Specs = [Spec|| Spec = {attribute,_,'spec',{FA,_}} <- AC], > ? ?io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(Specs))]). > > Hmm, just thinking. Maybe it would be a good idea to have the -spec > info available as a separate > chunk who is always present even without debug_info. > The -spec info could for example be used for a smarter tab-completion > of functions in the shell. > Including argument names and types. > > /Kenneth, Erlang/OTP Ericsson > > On Tue, Mar 8, 2011 at 3:41 PM, Eric Merritt wrote: >> Hello All, >> >> >> ?I have been doing some digging and it I haven't found anything that >> indicates that spec information survives compilation. It may in the >> beam files, but there isn't any documentation to support that >> assumption in beam_lib. So before I go digging in the source I thought >> I would ping the list and see if anyone has any ideas on accessing >> 'speced' type information at run time. >> >> Thanks, >> Eric >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From comptekki@REDACTED Tue Mar 8 18:14:21 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 8 Mar 2011 10:14:21 -0700 Subject: erlang statistics Message-ID: I'm going through some code in the book "Concurrent Programing in ERLANG". I ran the statistics on the sort and qsort thus: $ erl Erlang R14B01 (erts-5.8.2) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> statistics(exact_reductions), 1> sort:sort([2,1,4,23,6,7,8,43,9,4,7]), 1> {_, Reductions1} = statistics(exact_reductions), 1> qsort:qsort([2,1,4,23,6,7,8,43,9,4,7]), 1> {_, Reductions2} = statistics(exact_reductions), 1> io:format("~p~n",[Reductions1]), 1> io:format("~p~n",[Reductions2]). 3111 4922 ok I tried statistics(reductions) and statistics(exact_reductions) (per erlang.org docs). Why are the reductions different than the book and the qsort is slower than the sort, it appears?? thx, -wes From mjtruog@REDACTED Tue Mar 8 18:05:49 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 08 Mar 2011 09:05:49 -0800 Subject: [erlang-questions] Accessing 'spec' information at runtime In-Reply-To: References: Message-ID: <4D7661ED.8010500@gmail.com> There is already code for this: https://github.com/okeuday/CloudI/blob/master/src/lib/unused/src/abstract_code.erl An example is: 2> test:function(a). [{attribute,1,file,{"./test.erl",1}}, {attribute,1,module,test}, {attribute,4,export,[{function,1}]}, {attribute,6,spec, {{function,1}, [{type,6,'fun', [{type,6,product, [{ann_type,6,[{var,6,'A'},{type,6,any,[]}]}]}, {type,6,any,[]}]}]}}, {function,7,function,1, [{clause,7, [{var,7,'A'}], [], [{call,8, {remote,8,{atom,8,io},{atom,8,format}}, [{string,8,"~p~n"}, {cons,8, {call,8,{atom,8,'$abstract_code$'},[]}, {nil,8}}]}, {var,9,'A'}]}]}, {eof,10}] For the code: -module(test). -compile({parse_transform, abstract_code}). -export([function/1]). -spec function(A::any()) -> any(). function(A) -> io:format("~p~n", ['$abstract_code$'()]), A. On 03/08/2011 08:00 AM, Eric Merritt wrote: > I think that is an extraordinarily good idea Kenneth. Having that > information be a first class chunk would be useful. > > How do record type specs appear. I suspect they are part of the > abstract code for the record. > > On Tue, Mar 8, 2011 at 9:48 AM, Kenneth Lundin wrote: >> The -spec and -type is included in the abstract_code chunk in the beam >> files. If >> they are compiled to contain debug_info (erlc +debug_info my_module.erl). >> >> The appear as attributes and the can be abstracted like this (only the >> -spec(s) in this example): >> >> %% Beam is a string contaning the path to a beam file i.e >> "/home/user/mymodule.beam" >> %% you can for example call it like this from the shell: >> %% m:getspecs(code:which(lists)). >> >> getspecs(Beam)-> >> {ok,{_,[{abstract_code,{_,AC}}]}} = >> beam_lib:chunks(Beam,[abstract_code]), >> Specs = [Spec|| Spec = {attribute,_,'spec',{FA,_}} <- AC], >> io:fwrite("~s~n", [erl_prettypr:format(erl_syntax:form_list(Specs))]). >> >> Hmm, just thinking. Maybe it would be a good idea to have the -spec >> info available as a separate >> chunk who is always present even without debug_info. >> The -spec info could for example be used for a smarter tab-completion >> of functions in the shell. >> Including argument names and types. >> >> /Kenneth, Erlang/OTP Ericsson >> >> On Tue, Mar 8, 2011 at 3:41 PM, Eric Merritt wrote: >>> Hello All, >>> >>> >>> I have been doing some digging and it I haven't found anything that >>> indicates that spec information survives compilation. It may in the >>> beam files, but there isn't any documentation to support that >>> assumption in beam_lib. So before I go digging in the source I thought >>> I would ping the list and see if anyone has any ideas on accessing >>> 'speced' type information at run time. >>> >>> Thanks, >>> Eric >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dhananjay.nene@REDACTED Tue Mar 8 22:18:49 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Wed, 9 Mar 2011 02:48:49 +0530 Subject: Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children Message-ID: On Tue, Mar 8, 2011 at 4:13 PM, Edmond Begumisa wrote: > On Tue, 08 Mar 2011 17:52:29 +1100, Dhananjay Nene > wrote: > >> On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa >> wrote: >>> >>> Hi Dhananjay, >>> >>> I too struggled with this exact question for quite some time so I'll >>> chime >>> in here on the two techniques I used to solve it... >>> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >>> wrote: >>> >>>> >>>> Question in short : If I have a supervisor which has a number of >>>> dynamic children, how do I set up a mechanism where in case of a >>>> complete system crash, all the dynamic children restart at the point >>>> they were when the system (including the supervisor) crashed. >>>> >>>> Question in long : >>>> ============= >>>> >>>> Sample Context : A bowling game >>>> ------------------------------------------------- >>>> >>>> Lets say I am writing the software to implement the software necessary >>>> to track various games at a bowling alley. I've set up the following >>>> processes : >>>> >>>> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >>>> lane. These stay fixed for the entire duration of the program >>>> b. Games : A group of players might get together to start a game on a >>>> free lane. A new game will get created to track the game through its >>>> completion. When the game is over, this process shall terminate >>>> c. Players : Each game has a number of players. One process >>>> "player_game" is started per player. Sample state of a player game >>>> would include current score for the player and if the last two rolls >>>> were strike or a spare. For the purpose of brevity, the remainder of >>>> this mail only refers to this process and ignores the others >>>> >>> >>> You could reduce complexity by having each lane process maintain it's >>> current game (players and scores) as part of it's state. The game and >>> player_game processes appear unnecessarily confusing to me. >>> >> >> Interesting point. The lanes are the only static aspects of the game. >> I tried to consider whether it would make any difference from a client >> API perspective, but I imagine for a client, there is no particular >> reason to believe a lane is a better or worse abstraction than a game >> (or a player_game). >> >>>> Objective : >>>> --------------- >>>> >>>> Assuming this is a single node implementation, if the machine were to >>>> crash, upon machine / node restart, all the player_games should be >>>> restarted and should be at the point where the player_games were when >>>> the machine crashed. >>>> >>>> Possible supervision strategy : >>>> -------------------------------------- >>>> >>>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>>> starting up for the first time would have no children associated with >>>> them. Use supervisor:start_child to start each process >>>> 2. The supervisor creates an entry in a database (say mnesia) every >>>> time it launches a new process >>>> 3. Each player_game updates the entry every time the score gets >>>> modified. Upon termination that entry gets deleted >>>> 4. Post crash, the supervisor is started again (say after an >>>> application restart or via another supervisor) >>>> 5. (Here's the difference). By default the supervisor will not restart >>>> the dynamically added children (all the player_games). However we >>>> modify the init code to inspect the database and launch a player_game >>>> for each record it finds. >>> >>> How? I don't think you can instruct a simple_one_for_one supervisor to >>> create children from it's init/1 callback. From the documentation... >>> >>> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >>> >>> "...No child process is then started during the initialization phase, but >>> all children are assumed to be started dynamically using >>> supervisor:start_child/2..." >> >> Fair point. Wasn't something that struck me as an issue then, but yes, >> supervisor starting dynamic children inside init doesn't quite rock. >> >>> AFIAK, creating dynamic children (calling supervisor:start_child/2) has >>> to >>> be done after the supervisor has initialised by a process other than the >>> supervisor process. >> >> Certainly. And your separate modeling of a lane_ldr (later down this >> mail) helps that. >> >>> This is normally not a problem if you are calling start_child/2 during >>> the >>> "normal" operation of the application because the supervisor in question >>> is >>> likely to already be up. But here, you want to call start_child/2 at >>> *startup*. From my experience with this precise matter, this requires >>> some >>> process coordination. >>> >>>> The player_game initialises itself to the >>>> current state as in the database and the game(s) can continue where >>>> it/they left off. >>>> >>>> My questions : >>>> -------------------- >>>> a. Does it make sense to move the responsibility to the supervisor to >>>> update the database each time a new player game is started or >>>> completed ? >>> >>> I personally don't see the advantage of doing this. Besides (as per my >>> understanding of OTP design principles), a supervisor's job should be >>> just >>> that -- supervising workers and not doing work itself. >>> >>> Doing this from the your worker gen_servers make more sense to me and >>> seems >>> more natural. i.e Reading the scores from the DB the during >>> player_game:init >>> and writing them every time a score gets bumped or something similar. >>> >> >> I agree >> >> >>> Possible supervision strategy 2a: (Loader version) >>> -------------------------------------------------- >>> >>> Rather than separate dynamic children for players and games as in >>> Strategy >>> 1, instead, each lane stores, as part of it's state, info on the current >>> game (the players playing on the lane and their state/scores). The >>> supervision tree might look like this... >>> >>> ? ? ? ? ?alley_sup >>> ? ? ? ? / ? ? ? ? \ >>> ?lane_ldr ?___lanes_sup_____ >>> ? ? ? ? ?/ ? ? ? | ? ? : ? \ >>> ? ? ? lane(1) ?lane(2) .. lane(N) >>> >>> * Application has a startup configuration parameter no_of_lanes which >>> comes >>> from a conf file or the .app file and loaded by the alley_sup... >>> >> >> This is a suggestion thats really had me thinking. I suspect there's a >> bit of the traditional OO modeling experience which is grumbling about >> not being able to model a game or a player game. > > It's not that you can't model them, it's that you don't need to. > > One mantra in Erlang literature (e.g. Casarini & Thompson, pg110), is to > create a process for every concurrent *activity* you observe in the real > world and not every *task* you observe. So you don't necessarily need to use > a process for every "object" you see in the real world. > > With this in mind, my immediate interpretation of your application was in > two ways: > > A) > > * You have a bowling alley which has lanes. > * Different _lanes_ can be *concurrently* used at the same time: map these > to processes. > * Only 1 player can use a _lane_ at a time: no need for player processes. > * Only 1 game can take place on a _lane_ at a time: no need for game > processes. > * It follows that players and their game are just the state of each > concurrently used _lane_. > > So you only need processes for lanes. > > Alternatively, B) > > * You have a bowling alley where people play games. > * Several _games_ can be *concurrently* played at the same time: map these > to processes. > * Only 1 player can make a _game_ play at a time: no need for player > processes. > * Only 1 lane can be used per _game_: no need for lane processes. > * It follows that players and their lane are just the state of each > concurrently played game. > > So you only need processes for games. > > A) *might* be easier to implement than B) when you have to interact with > hardware that manages the lane machinery, which is why I suggested it. But > either way, you only need *one* class of processes. IMO, introducing more > just complicates matters unnecessarily. Understood. Here's a dump of my thoughts. Yes, we need to have upto only as many processes as the number of lanes (either the lanes themselves or maximum one game per lane). Lets assume we model a game as a process. There's a lot of state thats maintained at a player level, and very little at a game level. As a goal, separation of game and player (and the states) seems desirable to support these separation of concerns. Perhaps we could still model these using separate modules. Yet, given how state is carried over from one handle_call/handle_cast into the next, not modeling a player as a process forces the internal state of the player to be completely accessible to the game - most of it, it simply does not need access to. This stems from the fact that in erlang, state is maintained at a process level and not at a module level. I would imagine, this is a dilemma which is not entirely uncommon. Modeling a low level intricate module as a process, allows the finer details of the state to be contained in the implementation modules/processes, and can allow the policy to be maintained in a higher level module/process. This can also allow for easier way to change implementations (eg. hypothetically, in this case the precise semantics of scoring - one could go as far as defining a player to be behaviour). I am fully aware these are thoughts which stem from a classical OO experience. That begs the question -> are there situations where experienced erlang programmers choose to model processes not because they run concurrently, but because they have different independently encapsulated states, and in addition could also be helpful for separating out implementation specific behaviour. Or is there another programming feature / trick that I am not aware of which could help resolve these conflicting objectives? Dhananjay -- ----------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene From lukas.larsson@REDACTED Tue Mar 8 23:56:02 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Tue, 8 Mar 2011 22:56:02 +0000 (GMT) Subject: [erlang-questions] Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children In-Reply-To: Message-ID: <633946339.41631299624962567.JavaMail.root@zimbra> You can still do a logical separation between a player and a game, it does not have to be physical. By creating an opaque datatype for each player which you only can access through the player module you put a separation in between the two concepts but still keep the same concurrent activities as there are in the world. i.e. -module(player). %% Private record -record(player, {name}). create() -> #player{}. set_name(Name, Player) -> Player#player{ name = Name }. get_name(Player) -> Player#player.name. etc. This makes it (atleast for me) possible to enforce a logical model upon my physical restrictions which makes my code easier to read and also to maintain when upgrading. Lukas ----- Original Message ----- From: "Dhananjay Nene" To: erlang-questions@REDACTED Sent: Tuesday, March 8, 2011 10:18:49 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children On Tue, Mar 8, 2011 at 4:13 PM, Edmond Begumisa wrote: > On Tue, 08 Mar 2011 17:52:29 +1100, Dhananjay Nene > wrote: > >> On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa >> wrote: >>> >>> Hi Dhananjay, >>> >>> I too struggled with this exact question for quite some time so I'll >>> chime >>> in here on the two techniques I used to solve it... >>> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >>> wrote: >>> >>>> >>>> Question in short : If I have a supervisor which has a number of >>>> dynamic children, how do I set up a mechanism where in case of a >>>> complete system crash, all the dynamic children restart at the point >>>> they were when the system (including the supervisor) crashed. >>>> >>>> Question in long : >>>> ============= >>>> >>>> Sample Context : A bowling game >>>> ------------------------------------------------- >>>> >>>> Lets say I am writing the software to implement the software necessary >>>> to track various games at a bowling alley. I've set up the following >>>> processes : >>>> >>>> a. Lanes : If there are 10 lanes, there are 10 processes, one for each >>>> lane. These stay fixed for the entire duration of the program >>>> b. Games : A group of players might get together to start a game on a >>>> free lane. A new game will get created to track the game through its >>>> completion. When the game is over, this process shall terminate >>>> c. Players : Each game has a number of players. One process >>>> "player_game" is started per player. Sample state of a player game >>>> would include current score for the player and if the last two rolls >>>> were strike or a spare. For the purpose of brevity, the remainder of >>>> this mail only refers to this process and ignores the others >>>> >>> >>> You could reduce complexity by having each lane process maintain it's >>> current game (players and scores) as part of it's state. The game and >>> player_game processes appear unnecessarily confusing to me. >>> >> >> Interesting point. The lanes are the only static aspects of the game. >> I tried to consider whether it would make any difference from a client >> API perspective, but I imagine for a client, there is no particular >> reason to believe a lane is a better or worse abstraction than a game >> (or a player_game). >> >>>> Objective : >>>> --------------- >>>> >>>> Assuming this is a single node implementation, if the machine were to >>>> crash, upon machine / node restart, all the player_games should be >>>> restarted and should be at the point where the player_games were when >>>> the machine crashed. >>>> >>>> Possible supervision strategy : >>>> -------------------------------------- >>>> >>>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>>> starting up for the first time would have no children associated with >>>> them. Use supervisor:start_child to start each process >>>> 2. The supervisor creates an entry in a database (say mnesia) every >>>> time it launches a new process >>>> 3. Each player_game updates the entry every time the score gets >>>> modified. Upon termination that entry gets deleted >>>> 4. Post crash, the supervisor is started again (say after an >>>> application restart or via another supervisor) >>>> 5. (Here's the difference). By default the supervisor will not restart >>>> the dynamically added children (all the player_games). However we >>>> modify the init code to inspect the database and launch a player_game >>>> for each record it finds. >>> >>> How? I don't think you can instruct a simple_one_for_one supervisor to >>> create children from it's init/1 callback. From the documentation... >>> >>> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >>> >>> "...No child process is then started during the initialization phase, but >>> all children are assumed to be started dynamically using >>> supervisor:start_child/2..." >> >> Fair point. Wasn't something that struck me as an issue then, but yes, >> supervisor starting dynamic children inside init doesn't quite rock. >> >>> AFIAK, creating dynamic children (calling supervisor:start_child/2) has >>> to >>> be done after the supervisor has initialised by a process other than the >>> supervisor process. >> >> Certainly. And your separate modeling of a lane_ldr (later down this >> mail) helps that. >> >>> This is normally not a problem if you are calling start_child/2 during >>> the >>> "normal" operation of the application because the supervisor in question >>> is >>> likely to already be up. But here, you want to call start_child/2 at >>> *startup*. From my experience with this precise matter, this requires >>> some >>> process coordination. >>> >>>> The player_game initialises itself to the >>>> current state as in the database and the game(s) can continue where >>>> it/they left off. >>>> >>>> My questions : >>>> -------------------- >>>> a. Does it make sense to move the responsibility to the supervisor to >>>> update the database each time a new player game is started or >>>> completed ? >>> >>> I personally don't see the advantage of doing this. Besides (as per my >>> understanding of OTP design principles), a supervisor's job should be >>> just >>> that -- supervising workers and not doing work itself. >>> >>> Doing this from the your worker gen_servers make more sense to me and >>> seems >>> more natural. i.e Reading the scores from the DB the during >>> player_game:init >>> and writing them every time a score gets bumped or something similar. >>> >> >> I agree >> >> >>> Possible supervision strategy 2a: (Loader version) >>> -------------------------------------------------- >>> >>> Rather than separate dynamic children for players and games as in >>> Strategy >>> 1, instead, each lane stores, as part of it's state, info on the current >>> game (the players playing on the lane and their state/scores). The >>> supervision tree might look like this... >>> >>> ? ? ? ? ?alley_sup >>> ? ? ? ? / ? ? ? ? \ >>> ?lane_ldr ?___lanes_sup_____ >>> ? ? ? ? ?/ ? ? ? | ? ? : ? \ >>> ? ? ? lane(1) ?lane(2) .. lane(N) >>> >>> * Application has a startup configuration parameter no_of_lanes which >>> comes >>> from a conf file or the .app file and loaded by the alley_sup... >>> >> >> This is a suggestion thats really had me thinking. I suspect there's a >> bit of the traditional OO modeling experience which is grumbling about >> not being able to model a game or a player game. > > It's not that you can't model them, it's that you don't need to. > > One mantra in Erlang literature (e.g. Casarini & Thompson, pg110), is to > create a process for every concurrent *activity* you observe in the real > world and not every *task* you observe. So you don't necessarily need to use > a process for every "object" you see in the real world. > > With this in mind, my immediate interpretation of your application was in > two ways: > > A) > > * You have a bowling alley which has lanes. > * Different _lanes_ can be *concurrently* used at the same time: map these > to processes. > * Only 1 player can use a _lane_ at a time: no need for player processes. > * Only 1 game can take place on a _lane_ at a time: no need for game > processes. > * It follows that players and their game are just the state of each > concurrently used _lane_. > > So you only need processes for lanes. > > Alternatively, B) > > * You have a bowling alley where people play games. > * Several _games_ can be *concurrently* played at the same time: map these > to processes. > * Only 1 player can make a _game_ play at a time: no need for player > processes. > * Only 1 lane can be used per _game_: no need for lane processes. > * It follows that players and their lane are just the state of each > concurrently played game. > > So you only need processes for games. > > A) *might* be easier to implement than B) when you have to interact with > hardware that manages the lane machinery, which is why I suggested it. But > either way, you only need *one* class of processes. IMO, introducing more > just complicates matters unnecessarily. Understood. Here's a dump of my thoughts. Yes, we need to have upto only as many processes as the number of lanes (either the lanes themselves or maximum one game per lane). Lets assume we model a game as a process. There's a lot of state thats maintained at a player level, and very little at a game level. As a goal, separation of game and player (and the states) seems desirable to support these separation of concerns. Perhaps we could still model these using separate modules. Yet, given how state is carried over from one handle_call/handle_cast into the next, not modeling a player as a process forces the internal state of the player to be completely accessible to the game - most of it, it simply does not need access to. This stems from the fact that in erlang, state is maintained at a process level and not at a module level. I would imagine, this is a dilemma which is not entirely uncommon. Modeling a low level intricate module as a process, allows the finer details of the state to be contained in the implementation modules/processes, and can allow the policy to be maintained in a higher level module/process. This can also allow for easier way to change implementations (eg. hypothetically, in this case the precise semantics of scoring - one could go as far as defining a player to be behaviour). I am fully aware these are thoughts which stem from a classical OO experience. That begs the question -> are there situations where experienced erlang programmers choose to model processes not because they run concurrently, but because they have different independently encapsulated states, and in addition could also be helpful for separating out implementation specific behaviour. Or is there another programming feature / trick that I am not aware of which could help resolve these conflicting objectives? Dhananjay -- ----------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bernie@REDACTED Wed Mar 9 03:48:02 2011 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 09 Mar 2011 13:48:02 +1100 Subject: Binding variables from binaries in function headers Message-ID: <4D76EA62.1080608@m5net.com> So I've just run into an interesting little bit of behaviour that doesn't seem quite right. In the following code: ----------------------------------------- -module(casetest). -export([test/0]). test() -> match(<<1, 2, 3, 4, 5, 6, 7, 8>>). match(<>) -> case A of B -> wrong; _ -> ok end. ----------------------------------------- erlc gives me the warning ./casetest.erl:11: Warning: this clause cannot match because a previous clause at line 10 always matches (line 10 is the "B -> wrong;" line). And sure enough, if you run test/0 you get 'wrong' back. That, in itself, is curious to me since by my understanding B should be bound by the function header, and have no guarantee of being the same as A. I can't see how it could be unbound. Doubly curious, is that if I stop using B as the size specifier of C, like this: match(<>) -> The warning goes away. And the result becomes 'ok' (in spite of nothing in the body having changed, and the only thing changing in the header being the size of an unused variable at the tail of the binary). Similarly, if I change the body of match/1 to this: Z = B, case A of Z -> wrong; _ -> ok end. It also works. Bug? Feature? Me making an obvious blunder that nobody I've shown this to can spot? Cheers, Bernard From oribrost@REDACTED Wed Mar 9 04:20:26 2011 From: oribrost@REDACTED (ori brost) Date: Wed, 9 Mar 2011 05:20:26 +0200 Subject: Creating releases Message-ID: Is there any good tutorial explaining how to create releases for diskless nodes (i.e. there is also a bootserver running at a node that has a normal release)? All tutorials I found were about creating normal releases. Also, I'd prefer if the tutorial explained how to do these using reltool and rebar. From yujiangw@REDACTED Wed Mar 9 05:14:00 2011 From: yujiangw@REDACTED (Edward Wang) Date: Wed, 9 Mar 2011 12:14:00 +0800 Subject: [erlang-questions] Binding variables from binaries in function headers In-Reply-To: <4D76EA62.1080608@m5net.com> References: <4D76EA62.1080608@m5net.com> Message-ID: I can reproduce what you saw. And if I change the code as following: --------------------------- case A =:= B of true -> wrong; _ -> ok end. --------------------------- it also works. Very weird. For your original code erlc -S shows: ------------------------------------------- {move,{atom,wrong},{x,0}}. {deallocate,0}. return. ------------------------------------------- It optimizes your case clause away altogether! This looks like a bug to me. -Ed On Wed, Mar 9, 2011 at 10:48 AM, Bernard Duggan wrote: > So I've just run into an interesting little bit of behaviour that doesn't > seem quite right. In the following code: > ----------------------------------------- > -module(casetest). > > -export([test/0]). > > test() -> > match(<<1, 2, 3, 4, 5, 6, 7, 8>>). > > match(<>) -> > case A of > B -> wrong; > _ -> ok > end. > ----------------------------------------- > erlc gives me the warning > > ./casetest.erl:11: Warning: this clause cannot match because a previous > clause at line 10 always matches > > (line 10 is the "B -> wrong;" line). > > And sure enough, if you run test/0 you get 'wrong' back. > > That, in itself, is curious to me since by my understanding B should be > bound by the function header, and have no guarantee of being the same as A. > I can't see how it could be unbound. > > Doubly curious, is that if I stop using B as the size specifier of C, like > this: > > match(<>) -> > > The warning goes away. And the result becomes 'ok' (in spite of nothing in > the body having changed, and the only thing changing in the header being the > size of an unused variable at the tail of the binary). Similarly, if I > change the body of match/1 to this: > > Z = B, > case A of > Z -> wrong; > _ -> ok > end. > > It also works. > > Bug? Feature? Me making an obvious blunder that nobody I've shown this to > can spot? > > Cheers, > > Bernard > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Wed Mar 9 05:26:58 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 8 Mar 2011 21:26:58 -0700 Subject: [erlang-questions] Binding variables from binaries in function headers In-Reply-To: References: <4D76EA62.1080608@m5net.com> Message-ID: On Tue, Mar 8, 2011 at 7:48 PM, Bernard Duggan wrote: > So I've just run into an interesting little bit of behaviour that doesn't > seem quite right. In the following code: > ----------------------------------------- > -module(casetest). > > -export([test/0]). > > test() -> > match(<<1, 2, 3, 4, 5, 6, 7, 8>>). > > match(<>) -> > case A of > B -> wrong; > _ -> ok > end. I just ran it in the debugger: c(casetest,[debug_info]). im(). ii(casetest). iaa([init]). and it follows the _ -> ok branch and returns ok. Entering the function (the debugger stops at "case A of" ) the vars are: A is <<1>> B is 2 _C is <<3,4> _Rest is <<5,6,7,8>> -wes From dhananjay.nene@REDACTED Wed Mar 9 11:19:44 2011 From: dhananjay.nene@REDACTED (Dhananjay Nene) Date: Wed, 9 Mar 2011 15:49:44 +0530 Subject: [erlang-questions] Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children In-Reply-To: <633946339.41631299624962567.JavaMail.root@zimbra> References: <633946339.41631299624962567.JavaMail.root@zimbra> Message-ID: On Wed, Mar 9, 2011 at 4:26 AM, Lukas Larsson wrote: > You can still do a logical separation between a player and a game, it does not have to be physical. By creating an opaque datatype for each player which you only can access through the player module you put a separation in between the two concepts but still keep the same concurrent activities as there are in the world. i.e. > > -module(player). > %% Private record > -record(player, {name}). > > create() -> > ?#player{}. > > set_name(Name, Player) -> > ?Player#player{ name = Name }. > > get_name(Player) -> > ?Player#player.name. > > etc. > > This makes it (atleast for me) possible to enforce a logical model upon my physical restrictions which makes my code easier to read and also to maintain when upgrading. > > Lukas Thanks. That clarifies and helps. Dhananjay -- ----------------------------------------------------------------------------------- http://blog.dhananjaynene.com twitter: @dnene From ebegumisa@REDACTED Wed Mar 9 12:03:07 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 09 Mar 2011 22:03:07 +1100 Subject: [erlang-questions] Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children In-Reply-To: <633946339.41631299624962567.JavaMail.root@zimbra> References: <633946339.41631299624962567.JavaMail.root@zimbra> Message-ID: On Wed, 09 Mar 2011 09:56:02 +1100, Lukas Larsson wrote: > You can still do a logical separation between a player and a game, it > does not have to be physical. By creating an opaque datatype for each > player which you only can access through the player module you put a > separation in between the two concepts but still keep the same > concurrent activities as there are in the world. i.e. > > -module(player). > %% Private record > -record(player, {name}). > > create() -> > #player{}. > > set_name(Name, Player) -> > Player#player{ name = Name }. > > get_name(Player) -> > Player#player.name. > > etc. > > This makes it (atleast for me) possible to enforce a logical model upon > my physical restrictions which makes my code easier to read and also to > maintain when upgrading. > > Lukas > ----- Original Message ----- > From: "Dhananjay Nene" > To: erlang-questions@REDACTED > Sent: Tuesday, March 8, 2011 10:18:49 PM GMT +01:00 Amsterdam / Berlin / > Bern / Rome / Stockholm / Vienna > Subject: [erlang-questions] Granularity of process (and module) > identification. Was: Supervision strategies to automatically restart > dynamically added children > > On Tue, Mar 8, 2011 at 4:13 PM, Edmond Begumisa > wrote: >> On Tue, 08 Mar 2011 17:52:29 +1100, Dhananjay Nene >> wrote: >> >>> On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa >>> wrote: >>>> >>>> Hi Dhananjay, >>>> >>>> I too struggled with this exact question for quite some time so I'll >>>> chime >>>> in here on the two techniques I used to solve it... >>>> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >>>> wrote: >>>> >>>>> >>>>> Question in short : If I have a supervisor which has a number of >>>>> dynamic children, how do I set up a mechanism where in case of a >>>>> complete system crash, all the dynamic children restart at the point >>>>> they were when the system (including the supervisor) crashed. >>>>> >>>>> Question in long : >>>>> ============= >>>>> >>>>> Sample Context : A bowling game >>>>> ------------------------------------------------- >>>>> >>>>> Lets say I am writing the software to implement the software >>>>> necessary >>>>> to track various games at a bowling alley. I've set up the following >>>>> processes : >>>>> >>>>> a. Lanes : If there are 10 lanes, there are 10 processes, one for >>>>> each >>>>> lane. These stay fixed for the entire duration of the program >>>>> b. Games : A group of players might get together to start a game on a >>>>> free lane. A new game will get created to track the game through its >>>>> completion. When the game is over, this process shall terminate >>>>> c. Players : Each game has a number of players. One process >>>>> "player_game" is started per player. Sample state of a player game >>>>> would include current score for the player and if the last two rolls >>>>> were strike or a spare. For the purpose of brevity, the remainder of >>>>> this mail only refers to this process and ignores the others >>>>> >>>> >>>> You could reduce complexity by having each lane process maintain it's >>>> current game (players and scores) as part of it's state. The game and >>>> player_game processes appear unnecessarily confusing to me. >>>> >>> >>> Interesting point. The lanes are the only static aspects of the game. >>> I tried to consider whether it would make any difference from a client >>> API perspective, but I imagine for a client, there is no particular >>> reason to believe a lane is a better or worse abstraction than a game >>> (or a player_game). >>> >>>>> Objective : >>>>> --------------- >>>>> >>>>> Assuming this is a single node implementation, if the machine were to >>>>> crash, upon machine / node restart, all the player_games should be >>>>> restarted and should be at the point where the player_games were when >>>>> the machine crashed. >>>>> >>>>> Possible supervision strategy : >>>>> -------------------------------------- >>>>> >>>>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>>>> starting up for the first time would have no children associated with >>>>> them. Use supervisor:start_child to start each process >>>>> 2. The supervisor creates an entry in a database (say mnesia) every >>>>> time it launches a new process >>>>> 3. Each player_game updates the entry every time the score gets >>>>> modified. Upon termination that entry gets deleted >>>>> 4. Post crash, the supervisor is started again (say after an >>>>> application restart or via another supervisor) >>>>> 5. (Here's the difference). By default the supervisor will not >>>>> restart >>>>> the dynamically added children (all the player_games). However we >>>>> modify the init code to inspect the database and launch a player_game >>>>> for each record it finds. >>>> >>>> How? I don't think you can instruct a simple_one_for_one supervisor to >>>> create children from it's init/1 callback. From the documentation... >>>> >>>> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >>>> >>>> "...No child process is then started during the initialization phase, >>>> but >>>> all children are assumed to be started dynamically using >>>> supervisor:start_child/2..." >>> >>> Fair point. Wasn't something that struck me as an issue then, but yes, >>> supervisor starting dynamic children inside init doesn't quite rock. >>> >>>> AFIAK, creating dynamic children (calling supervisor:start_child/2) >>>> has >>>> to >>>> be done after the supervisor has initialised by a process other than >>>> the >>>> supervisor process. >>> >>> Certainly. And your separate modeling of a lane_ldr (later down this >>> mail) helps that. >>> >>>> This is normally not a problem if you are calling start_child/2 during >>>> the >>>> "normal" operation of the application because the supervisor in >>>> question >>>> is >>>> likely to already be up. But here, you want to call start_child/2 at >>>> *startup*. From my experience with this precise matter, this requires >>>> some >>>> process coordination. >>>> >>>>> The player_game initialises itself to the >>>>> current state as in the database and the game(s) can continue where >>>>> it/they left off. >>>>> >>>>> My questions : >>>>> -------------------- >>>>> a. Does it make sense to move the responsibility to the supervisor to >>>>> update the database each time a new player game is started or >>>>> completed ? >>>> >>>> I personally don't see the advantage of doing this. Besides (as per my >>>> understanding of OTP design principles), a supervisor's job should be >>>> just >>>> that -- supervising workers and not doing work itself. >>>> >>>> Doing this from the your worker gen_servers make more sense to me and >>>> seems >>>> more natural. i.e Reading the scores from the DB the during >>>> player_game:init >>>> and writing them every time a score gets bumped or something similar. >>>> >>> >>> I agree >>> >>> >>>> Possible supervision strategy 2a: (Loader version) >>>> -------------------------------------------------- >>>> >>>> Rather than separate dynamic children for players and games as in >>>> Strategy >>>> 1, instead, each lane stores, as part of it's state, info on the >>>> current >>>> game (the players playing on the lane and their state/scores). The >>>> supervision tree might look like this... >>>> >>>> alley_sup >>>> / \ >>>> lane_ldr ___lanes_sup_____ >>>> / | : \ >>>> lane(1) lane(2) .. lane(N) >>>> >>>> * Application has a startup configuration parameter no_of_lanes which >>>> comes >>>> from a conf file or the .app file and loaded by the alley_sup... >>>> >>> >>> This is a suggestion thats really had me thinking. I suspect there's a >>> bit of the traditional OO modeling experience which is grumbling about >>> not being able to model a game or a player game. >> >> It's not that you can't model them, it's that you don't need to. >> >> One mantra in Erlang literature (e.g. Casarini & Thompson, pg110), is to >> create a process for every concurrent *activity* you observe in the real >> world and not every *task* you observe. So you don't necessarily need >> to use >> a process for every "object" you see in the real world. >> >> With this in mind, my immediate interpretation of your application was >> in >> two ways: >> >> A) >> >> * You have a bowling alley which has lanes. >> * Different _lanes_ can be *concurrently* used at the same time: map >> these >> to processes. >> * Only 1 player can use a _lane_ at a time: no need for player >> processes. >> * Only 1 game can take place on a _lane_ at a time: no need for game >> processes. >> * It follows that players and their game are just the state of each >> concurrently used _lane_. >> >> So you only need processes for lanes. >> >> Alternatively, B) >> >> * You have a bowling alley where people play games. >> * Several _games_ can be *concurrently* played at the same time: map >> these >> to processes. >> * Only 1 player can make a _game_ play at a time: no need for player >> processes. >> * Only 1 lane can be used per _game_: no need for lane processes. >> * It follows that players and their lane are just the state of each >> concurrently played game. >> >> So you only need processes for games. >> >> A) *might* be easier to implement than B) when you have to interact with >> hardware that manages the lane machinery, which is why I suggested it. >> But >> either way, you only need *one* class of processes. IMO, introducing >> more >> just complicates matters unnecessarily. > > Understood. Here's a dump of my thoughts. > > Yes, we need to have upto only as many processes as the number of > lanes (either the lanes themselves or maximum one game per lane). Lets > assume we model a game as a process. > > There's a lot of state thats maintained at a player level, and very > little at a game level. As a goal, separation of game and player (and > the states) seems desirable to support these separation of concerns. > Perhaps we could still model these using separate modules. Yet, given > how state is carried over from one handle_call/handle_cast into the > next, not modeling a player as a process forces the internal state of > the player to be completely accessible to the game - most of it, it > simply does not need access to. Not necessarily. To reinforce what Lukas said above, it could treat the player and game elements of it's state as something it doesn't understand but keeps hold of. It then takes actions on these elements by calling other modules player.erl and game.erl that do understand them. In this case, those modules would not need to create their own processes to have some seperation. I'll expand on Lukas's example coz I've found it can be hard to visualise if you've come from OO... === lane.erl === -behaviour(gen_server). .. init(Id) -> .. % We don't know what we're creating, we just know it represents a game {ok, game:create(Id)}. hanlde_cast({add_player, PlayerName}, Game0) -> {noreply, game:add_player(Game0)}; handle_cast({game_play, PlayerName, PinsDown}, Game0) -> % game:play must succeed otherwise lane/game state will be incorrect Game1 = game:play(Game0, PlayerName, PinsDown), ok = reset_pins(), {noreply, Game1}. handle_call({get_score, PlayerName}, Game) -> case Reply = game:get_score(Game, PlayerName) of {ok, Score} -> {reply, Reply, Game}. Error -> {reply, Reply, Game} % Same but just making it clear that game:get_score is allowed to fail end. reset_pins() -> .. maybe talk to some hardware driver .. == game.erl=== -export(create/1, add_player/2, play/3). create(LaneId) -> .. create path from LaneId .. AllPlayers = try read_game(PersistPath) catch _:Why -> .. [] end {Id,PersistPath,AllPlayers}. % Understood only by game.erl add_player({LaneId,PersistPath,AllPlayers0}, PlayerName) -> {LaneId, PersistPath, lists:keystore(PlayerName, 1, AllPlayers0, player:create(PlayerName)). %% Part of Jasper's error kernel -- assert happy-case play({LaneId,PersistPath,AllPlayers0}, PlayerName, PinsDown) -> Player0 = proplist:get_value(PlayerName, AllPlayers0), false = undefined == Player0, AllPlayers1 = lists:keyreplace(PlayerName, 1, AllPlayers0, player:bump_score(Player0, PinsDown))}, ok = write_game(PersistPath, AllPlayers1), {LaneId,PersistPath,AllPlayers1}. get_score({_,_,AllPlayers), PlayerName) -> case proplist:get_value(PlayerName, AllPlayers) of undefined -> {badarg, PlayerName}; Player -> {ok, player:get_score(Player)} end. %%% Internal functions read_game() -> .. write_game() -> .. == player.erl === -export(create, bump_score, get_score) %% Jasper's Error Kernel: This state and operations against it must be correct. -record(state, {frame = 0, shot = 1, bonus_shot = false, last_shot = normal, prior_to_last_shot = normal, max_pins = 10, score = 0}). create() -> #state{}. % Only understood by player.erl bump_score(#state{} = Player, PinsDown) -> .. get_score(#state{} = Player) -> .. > This stems from the fact that in > erlang, state is maintained at a process level and not at a module > level. > > I would imagine, this is a dilemma which is not entirely uncommon. > Modeling a low level intricate module as a process, allows the finer > details of the state to be contained in the implementation > modules/processes, and can allow the policy to be maintained in a > higher level module/process. This can also allow for easier way to > change implementations (eg. hypothetically, in this case the precise > semantics of scoring - one could go as far as defining a player to be > behaviour). I am fully aware these are thoughts which stem from a > classical OO experience. > > That begs the question -> are there situations where experienced > erlang programmers choose to model processes not because they run > concurrently, but because they have different independently > encapsulated states, and in addition could also be helpful for > separating out implementation specific behaviour. [Keeping in mind that I'm not an "experienced Erlang programmer"] State machines come to mind (gen_fsm). You could certainly go your original route. Process creation is fast. Message passing is fast. Just be careful not to inadvertently create a N-to-1-to-N routing for messages between processes. This can become a bottleneck when a program grows and/or becomes very busy. Tracing and debugging gets a little tricker too. - Edmond - It can also make things a little hard to follow in terms of tracing and debugging. > Or is there another > programming feature / trick that I am not aware of which could help > resolve these conflicting objectives? > > Dhananjay > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Wed Mar 9 13:03:43 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 09 Mar 2011 23:03:43 +1100 Subject: [erlang-questions] Granularity of process (and module) identification. Was: Supervision strategies to automatically restart dynamically added children In-Reply-To: References: <633946339.41631299624962567.JavaMail.root@zimbra> Message-ID: Forgot the all important new_game... On Wed, 09 Mar 2011 22:03:07 +1100, Edmond Begumisa wrote: > On Wed, 09 Mar 2011 09:56:02 +1100, Lukas Larsson > wrote: > >> You can still do a logical separation between a player and a game, it >> does not have to be physical. By creating an opaque datatype for each >> player which you only can access through the player module you put a >> separation in between the two concepts but still keep the same >> concurrent activities as there are in the world. i.e. >> >> -module(player). >> %% Private record >> -record(player, {name}). >> >> create() -> >> #player{}. >> >> set_name(Name, Player) -> >> Player#player{ name = Name }. >> >> get_name(Player) -> >> Player#player.name. >> >> etc. >> >> This makes it (atleast for me) possible to enforce a logical model upon >> my physical restrictions which makes my code easier to read and also to >> maintain when upgrading. >> >> Lukas >> ----- Original Message ----- >> From: "Dhananjay Nene" >> To: erlang-questions@REDACTED >> Sent: Tuesday, March 8, 2011 10:18:49 PM GMT +01:00 Amsterdam / Berlin >> / Bern / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Granularity of process (and module) >> identification. Was: Supervision strategies to automatically restart >> dynamically added children >> >> On Tue, Mar 8, 2011 at 4:13 PM, Edmond Begumisa >> wrote: >>> On Tue, 08 Mar 2011 17:52:29 +1100, Dhananjay Nene >>> wrote: >>> >>>> On Mon, Mar 7, 2011 at 5:08 AM, Edmond Begumisa >>>> wrote: >>>>> >>>>> Hi Dhananjay, >>>>> >>>>> I too struggled with this exact question for quite some time so I'll >>>>> chime >>>>> in here on the two techniques I used to solve it... >>>>> On Thu, 03 Mar 2011 05:02:06 +1100, Dhananjay Nene >>>>> wrote: >>>>> >>>>>> >>>>>> Question in short : If I have a supervisor which has a number of >>>>>> dynamic children, how do I set up a mechanism where in case of a >>>>>> complete system crash, all the dynamic children restart at the point >>>>>> they were when the system (including the supervisor) crashed. >>>>>> >>>>>> Question in long : >>>>>> ============= >>>>>> >>>>>> Sample Context : A bowling game >>>>>> ------------------------------------------------- >>>>>> >>>>>> Lets say I am writing the software to implement the software >>>>>> necessary >>>>>> to track various games at a bowling alley. I've set up the following >>>>>> processes : >>>>>> >>>>>> a. Lanes : If there are 10 lanes, there are 10 processes, one for >>>>>> each >>>>>> lane. These stay fixed for the entire duration of the program >>>>>> b. Games : A group of players might get together to start a game on >>>>>> a >>>>>> free lane. A new game will get created to track the game through its >>>>>> completion. When the game is over, this process shall terminate >>>>>> c. Players : Each game has a number of players. One process >>>>>> "player_game" is started per player. Sample state of a player game >>>>>> would include current score for the player and if the last two rolls >>>>>> were strike or a spare. For the purpose of brevity, the remainder of >>>>>> this mail only refers to this process and ignores the others >>>>>> >>>>> >>>>> You could reduce complexity by having each lane process maintain it's >>>>> current game (players and scores) as part of it's state. The game and >>>>> player_game processes appear unnecessarily confusing to me. >>>>> >>>> >>>> Interesting point. The lanes are the only static aspects of the game. >>>> I tried to consider whether it would make any difference from a client >>>> API perspective, but I imagine for a client, there is no particular >>>> reason to believe a lane is a better or worse abstraction than a game >>>> (or a player_game). >>>> >>>>>> Objective : >>>>>> --------------- >>>>>> >>>>>> Assuming this is a single node implementation, if the machine were >>>>>> to >>>>>> crash, upon machine / node restart, all the player_games should be >>>>>> restarted and should be at the point where the player_games were >>>>>> when >>>>>> the machine crashed. >>>>>> >>>>>> Possible supervision strategy : >>>>>> -------------------------------------- >>>>>> >>>>>> 1. Create a simple_one_for_one supervisor player_game_sup which upon >>>>>> starting up for the first time would have no children associated >>>>>> with >>>>>> them. Use supervisor:start_child to start each process >>>>>> 2. The supervisor creates an entry in a database (say mnesia) every >>>>>> time it launches a new process >>>>>> 3. Each player_game updates the entry every time the score gets >>>>>> modified. Upon termination that entry gets deleted >>>>>> 4. Post crash, the supervisor is started again (say after an >>>>>> application restart or via another supervisor) >>>>>> 5. (Here's the difference). By default the supervisor will not >>>>>> restart >>>>>> the dynamically added children (all the player_games). However we >>>>>> modify the init code to inspect the database and launch a >>>>>> player_game >>>>>> for each record it finds. >>>>> >>>>> How? I don't think you can instruct a simple_one_for_one supervisor >>>>> to >>>>> create children from it's init/1 callback. From the documentation... >>>>> >>>>> http://www.erlang.org/doc/man/supervisor.html#Module:init-1 >>>>> >>>>> "...No child process is then started during the initialization >>>>> phase, but >>>>> all children are assumed to be started dynamically using >>>>> supervisor:start_child/2..." >>>> >>>> Fair point. Wasn't something that struck me as an issue then, but yes, >>>> supervisor starting dynamic children inside init doesn't quite rock. >>>> >>>>> AFIAK, creating dynamic children (calling supervisor:start_child/2) >>>>> has >>>>> to >>>>> be done after the supervisor has initialised by a process other than >>>>> the >>>>> supervisor process. >>>> >>>> Certainly. And your separate modeling of a lane_ldr (later down this >>>> mail) helps that. >>>> >>>>> This is normally not a problem if you are calling start_child/2 >>>>> during >>>>> the >>>>> "normal" operation of the application because the supervisor in >>>>> question >>>>> is >>>>> likely to already be up. But here, you want to call start_child/2 at >>>>> *startup*. From my experience with this precise matter, this requires >>>>> some >>>>> process coordination. >>>>> >>>>>> The player_game initialises itself to the >>>>>> current state as in the database and the game(s) can continue where >>>>>> it/they left off. >>>>>> >>>>>> My questions : >>>>>> -------------------- >>>>>> a. Does it make sense to move the responsibility to the supervisor >>>>>> to >>>>>> update the database each time a new player game is started or >>>>>> completed ? >>>>> >>>>> I personally don't see the advantage of doing this. Besides (as per >>>>> my >>>>> understanding of OTP design principles), a supervisor's job should be >>>>> just >>>>> that -- supervising workers and not doing work itself. >>>>> >>>>> Doing this from the your worker gen_servers make more sense to me and >>>>> seems >>>>> more natural. i.e Reading the scores from the DB the during >>>>> player_game:init >>>>> and writing them every time a score gets bumped or something similar. >>>>> >>>> >>>> I agree >>>> >>>> >>>>> Possible supervision strategy 2a: (Loader version) >>>>> -------------------------------------------------- >>>>> >>>>> Rather than separate dynamic children for players and games as in >>>>> Strategy >>>>> 1, instead, each lane stores, as part of it's state, info on the >>>>> current >>>>> game (the players playing on the lane and their state/scores). The >>>>> supervision tree might look like this... >>>>> >>>>> alley_sup >>>>> / \ >>>>> lane_ldr ___lanes_sup_____ >>>>> / | : \ >>>>> lane(1) lane(2) .. lane(N) >>>>> >>>>> * Application has a startup configuration parameter no_of_lanes which >>>>> comes >>>>> from a conf file or the .app file and loaded by the alley_sup... >>>>> >>>> >>>> This is a suggestion thats really had me thinking. I suspect there's a >>>> bit of the traditional OO modeling experience which is grumbling about >>>> not being able to model a game or a player game. >>> >>> It's not that you can't model them, it's that you don't need to. >>> >>> One mantra in Erlang literature (e.g. Casarini & Thompson, pg110), is >>> to >>> create a process for every concurrent *activity* you observe in the >>> real >>> world and not every *task* you observe. So you don't necessarily need >>> to use >>> a process for every "object" you see in the real world. >>> >>> With this in mind, my immediate interpretation of your application was >>> in >>> two ways: >>> >>> A) >>> >>> * You have a bowling alley which has lanes. >>> * Different _lanes_ can be *concurrently* used at the same time: map >>> these >>> to processes. >>> * Only 1 player can use a _lane_ at a time: no need for player >>> processes. >>> * Only 1 game can take place on a _lane_ at a time: no need for game >>> processes. >>> * It follows that players and their game are just the state of each >>> concurrently used _lane_. >>> >>> So you only need processes for lanes. >>> >>> Alternatively, B) >>> >>> * You have a bowling alley where people play games. >>> * Several _games_ can be *concurrently* played at the same time: map >>> these >>> to processes. >>> * Only 1 player can make a _game_ play at a time: no need for player >>> processes. >>> * Only 1 lane can be used per _game_: no need for lane processes. >>> * It follows that players and their lane are just the state of each >>> concurrently played game. >>> >>> So you only need processes for games. >>> >>> A) *might* be easier to implement than B) when you have to interact >>> with >>> hardware that manages the lane machinery, which is why I suggested it. >>> But >>> either way, you only need *one* class of processes. IMO, introducing >>> more >>> just complicates matters unnecessarily. >> >> Understood. Here's a dump of my thoughts. >> >> Yes, we need to have upto only as many processes as the number of >> lanes (either the lanes themselves or maximum one game per lane). Lets >> assume we model a game as a process. >> >> There's a lot of state thats maintained at a player level, and very >> little at a game level. As a goal, separation of game and player (and >> the states) seems desirable to support these separation of concerns. >> Perhaps we could still model these using separate modules. Yet, given >> how state is carried over from one handle_call/handle_cast into the >> next, not modeling a player as a process forces the internal state of >> the player to be completely accessible to the game - most of it, it >> simply does not need access to. > > Not necessarily. To reinforce what Lukas said above, it could treat the > player and game elements of it's state as something it doesn't > understand but keeps hold of. It then takes actions on these elements by > calling other modules player.erl and game.erl that do understand them. > In this case, those modules would not need to create their own processes > to have some seperation. > > I'll expand on Lukas's example coz I've found it can be hard to > visualise if you've come from OO... > > === lane.erl === > -behaviour(gen_server). > .. > init(Id) -> > .. > % We don't know what we're creating, we just know it represents a > game > {ok, game:create(Id)}. > hanlde_cast(new_game, OldGame) -> {noreply, game:reset(OldGame)}; > hanlde_cast({add_player, PlayerName}, Game0) -> > {noreply, game:add_player(Game0)}; > handle_cast({game_play, PlayerName, PinsDown}, Game0) -> > % game:play must succeed otherwise lane/game state will be incorrect > Game1 = game:play(Game0, PlayerName, PinsDown), > ok = reset_pins(), > {noreply, Game1}. > > handle_call({get_score, PlayerName}, Game) -> > case Reply = game:get_score(Game, PlayerName) of > {ok, Score} -> > {reply, Reply, Game}. > Error -> > {reply, Reply, Game} % Same but just making it clear that > game:get_score is allowed to fail > end. > > reset_pins() -> > .. maybe talk to some hardware driver .. > > == game.erl=== > -export(create/1, add_player/2, play/3). > > create(LaneId) -> > .. create path from LaneId .. > AllPlayers = try read_game(PersistPath) > catch > _:Why -> > .. > [] > end > {Id,PersistPath,AllPlayers}. % Understood only by game.erl > reset({LaneId,PersistPath,_}) -> ok = write_game(PersistPath, []), {LaneId,PersistPath,[]). > add_player({LaneId,PersistPath,AllPlayers0}, PlayerName) -> > {LaneId, PersistPath, lists:keystore(PlayerName, 1, AllPlayers0, > player:create(PlayerName)). > > %% Part of Jasper's error kernel -- assert happy-case > play({LaneId,PersistPath,AllPlayers0}, PlayerName, PinsDown) -> > Player0 = proplist:get_value(PlayerName, AllPlayers0), > false = undefined == Player0, > AllPlayers1 = lists:keyreplace(PlayerName, 1, AllPlayers0, > player:bump_score(Player0, > PinsDown))}, > ok = write_game(PersistPath, AllPlayers1), > {LaneId,PersistPath,AllPlayers1}. > > get_score({_,_,AllPlayers), PlayerName) -> > case proplist:get_value(PlayerName, AllPlayers) of > undefined -> > {badarg, PlayerName}; > Player -> > {ok, player:get_score(Player)} > end. > > %%% Internal functions > read_game() -> > .. > write_game() -> > .. > > == player.erl === > > -export(create, bump_score, get_score) > > %% Jasper's Error Kernel: This state and operations against it must be > correct. > -record(state, {frame = 0, > shot = 1, > bonus_shot = false, > last_shot = normal, > prior_to_last_shot = normal, > max_pins = 10, > score = 0}). > create() -> > #state{}. % Only understood by player.erl > > bump_score(#state{} = Player, PinsDown) -> > .. > get_score(#state{} = Player) -> > .. > > >> This stems from the fact that in >> erlang, state is maintained at a process level and not at a module >> level. >> >> I would imagine, this is a dilemma which is not entirely uncommon. >> Modeling a low level intricate module as a process, allows the finer >> details of the state to be contained in the implementation >> modules/processes, and can allow the policy to be maintained in a >> higher level module/process. This can also allow for easier way to >> change implementations (eg. hypothetically, in this case the precise >> semantics of scoring - one could go as far as defining a player to be >> behaviour). I am fully aware these are thoughts which stem from a >> classical OO experience. >> >> That begs the question -> are there situations where experienced >> erlang programmers choose to model processes not because they run >> concurrently, but because they have different independently >> encapsulated states, and in addition could also be helpful for >> separating out implementation specific behaviour. > > [Keeping in mind that I'm not an "experienced Erlang programmer"] > > State machines come to mind (gen_fsm). > > You could certainly go your original route. Process creation is fast. > Message passing is fast. Just be careful not to inadvertently create a > N-to-1-to-N routing for messages between processes. This can become a > bottleneck when a program grows and/or becomes very busy. Tracing and > debugging gets a little tricker too. > > - Edmond - > > It can also make things a little hard to follow in terms of tracing and > debugging. > >> Or is there another >> programming feature / trick that I am not aware of which could help >> resolve these conflicting objectives? >> >> Dhananjay >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From nox@REDACTED Wed Mar 9 14:59:43 2011 From: nox@REDACTED (Anthony Ramine) Date: Wed, 9 Mar 2011 14:59:43 +0100 Subject: erl_lint:pattern/5 documentation Message-ID: Hi, Isn't there a problem with erl_lint:pattern/5 documentation [1]? From what I understand, UpdVarTable and BinVarTable descriptions are reversed, aren't they? Regards, [1] https://github.com/erlang/otp/blob/dev/lib/stdlib/src/erl_lint.erl#L1361 -- Anthony Ramine Dev:Extend http://dev-extend.eu From nox@REDACTED Wed Mar 9 15:14:20 2011 From: nox@REDACTED (Anthony Ramine) Date: Wed, 9 Mar 2011 15:14:20 +0100 Subject: Structs? In my Erlang? Message-ID: <5D1E109A-5BD3-4671-856C-135B4FCDE709@dev-extend.eu> Hi, I've stumbled upon a small commented out Erlang feature, tagged tuples called structs [1]. After further investigation, I've found out some comments saying it's a future feature which hasn't been completed yet [2]. Where it gets interesting is that this comment was already there in R6B0, which was released in 1999 [3]. So is that a feature that should be finished, superseded by frames or should it be entirely removed? Please note that the commented out code isn't even up to date now [4]. [1] https://github.com/erlang/otp/blob/dev/lib/stdlib/src/erl_parse.yrl#L333 [2] https://github.com/erlang/otp/blob/dev/lib/compiler/src/sys_pre_expand.erl#L22 [3] https://github.com/mfoemmel/erlang-otp/blob/R6B-0/lib/compiler/src/sys_pre_expand.erl#L21 [4] https://github.com/erlang/otp/blob/dev/lib/stdlib/src/erl_expand_records.erl#L302 Regards, -- Anthony Ramine Dev:Extend http://dev-extend.eu From bob@REDACTED Wed Mar 9 15:21:19 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 09 Mar 2011 14:21:19 +0000 Subject: Bit syntax Message-ID: <4D778CDF.3090907@bobcowdery.plus.com> Hi I'm clearly not understanding the bit syntax properly. Could someone explain to me what I am doing wrong here. Why does the first not match when its the right shape but the second does when its the wrong shape. The bound values don't make much sense to me either. Thanks Bob 1> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. ** exception error: no match of right hand side value <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,...>> 2> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,...>> 3> A. 4026401280 4> B. 0 5> C. 408 From trevorw@REDACTED Wed Mar 9 15:27:50 2011 From: trevorw@REDACTED (Trevor Woollacott) Date: Wed, 9 Mar 2011 16:27:50 +0200 Subject: [erlang-questions] Bit syntax In-Reply-To: <4D778CDF.3090907@bobcowdery.plus.com> References: <4D778CDF.3090907@bobcowdery.plus.com> Message-ID: <051C0339DDDD45D389968F59D485612E@mtn.co.za> Hi It doesn't match in your first one because you have that extra byte (16#00) in the binary. The second one matches because A holds an extra 8 bits Regards, Trevor -------------------------------------------------- From: "Bob Cowdery" Sent: Wednesday, March 09, 2011 4:21 PM To: "erlang-questions" Subject: [erlang-questions] Bit syntax > Hi > > I'm clearly not understanding the bit syntax properly. Could someone > explain to me what I am doing wrong here. Why does the first not match > when its the right shape but the second does when its the wrong shape. > The bound values don't make much sense to me either. > > Thanks > Bob > > 1> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > ** exception error: no match of right hand side value > > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > 2> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > 3> A. > 4026401280 > 4> B. > 0 > 5> C. > 408 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vladdu55@REDACTED Wed Mar 9 15:29:41 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 9 Mar 2011 15:29:41 +0100 Subject: [erlang-questions] Bit syntax In-Reply-To: <4D778CDF.3090907@bobcowdery.plus.com> References: <4D778CDF.3090907@bobcowdery.plus.com> Message-ID: Hi! On Wed, Mar 9, 2011 at 15:21, Bob Cowdery wrote: > Hi > > I'm clearly not understanding the bit syntax properly. Could someone > explain to me what I am doing wrong here. Why does the first not match > when its the right shape but the second does when its the wrong shape. > The bound values don't make much sense to me either. > > 1> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > ** exception error: no match of right hand side value > There is an "extra" 16#00 on the right side that is not matched on the left. > 2> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > The larger size for A "swallows" the extra value, but shifts the values a byte to the right. regards, Vlad From bob@REDACTED Wed Mar 9 15:50:46 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 09 Mar 2011 14:50:46 +0000 Subject: [erlang-questions] Bit syntax In-Reply-To: References: <4D778CDF.3090907@bobcowdery.plus.com> Message-ID: <4D7793C6.7080903@bobcowdery.plus.com> Thanks. The syntax did look a bit odd for a repeating group. I pulled it off the web somewhere. So if I say: 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> . <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,...>> 8> A. 15728130 9> B. 0 10> C. 408 What do these bound values mean. C is supposed to be 51 zero bytes (408 bits). Why is C bound to the number of bits. Bob On 09/03/2011 14:29, Vlad Dumitrescu wrote: > Hi! > > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery > wrote: > > Hi > > I'm clearly not understanding the bit syntax properly. Could someone > explain to me what I am doing wrong here. Why does the first not match > when its the right shape but the second does when its the wrong shape. > The bound values don't make much sense to me either. > > 1> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > ** exception error: no match of right hand side value > > > There is an "extra" 16#00 on the right side that is not matched on the > left. > > > 2> <> = <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > > > The larger size for A "swallows" the extra value, but shifts the > values a byte to the right. > > regards, > Vlad > From vladdu55@REDACTED Wed Mar 9 15:55:10 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 9 Mar 2011 15:55:10 +0100 Subject: [erlang-questions] Bit syntax In-Reply-To: <4D7793C6.7080903@bobcowdery.plus.com> References: <4D778CDF.3090907@bobcowdery.plus.com> <4D7793C6.7080903@bobcowdery.plus.com> Message-ID: Your C is 8*51 as a 408 bit number. You probably want 16#00:408 instead. /Vlad On Wed, Mar 9, 2011 at 15:50, Bob Cowdery wrote: > Thanks. The syntax did look a bit odd for a repeating group. I pulled it > off the web somewhere. > > So if I say: > > 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> > . > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > 8> A. > 15728130 > 9> B. > 0 > 10> C. > 408 > > What do these bound values mean. C is supposed to be 51 zero bytes (408 > bits). Why is C bound to the number of bits. > > Bob > > On 09/03/2011 14:29, Vlad Dumitrescu wrote: > > Hi! > > > > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery > > wrote: > > > > Hi > > > > I'm clearly not understanding the bit syntax properly. Could someone > > explain to me what I am doing wrong here. Why does the first not > match > > when its the right shape but the second does when its the wrong > shape. > > The bound values don't make much sense to me either. > > > > 1> <> = > <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > > ** exception error: no match of right hand side value > > > > > > There is an "extra" 16#00 on the right side that is not matched on the > > left. > > > > > > 2> <> = > <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > > > > > > The larger size for A "swallows" the extra value, but shifts the > > values a byte to the right. > > > > regards, > > Vlad > > > > From bob@REDACTED Wed Mar 9 16:09:59 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 09 Mar 2011 15:09:59 +0000 Subject: [erlang-questions] Bit syntax In-Reply-To: References: <4D778CDF.3090907@bobcowdery.plus.com> <4D7793C6.7080903@bobcowdery.plus.com> Message-ID: <4D779847.3030601@bobcowdery.plus.com> What I actually want is 51 bytes of 16#00. 12> <> = <<16#effe02:24,16#00:48,16#00:408>>. ** exception error: no match of right hand side value <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,...>> Why is this different. Isn't it just saying 00 as a 408 bit number rather than (8*51)? Bob On 09/03/2011 14:55, Vlad Dumitrescu wrote: > Your C is 8*51 as a 408 bit number. You probably want 16#00:408 instead. > > /Vlad > > On Wed, Mar 9, 2011 at 15:50, Bob Cowdery > wrote: > > Thanks. The syntax did look a bit odd for a repeating group. I > pulled it > off the web somewhere. > > So if I say: > > 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> > . > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > 8> A. > 15728130 > 9> B. > 0 > 10> C. > 408 > > What do these bound values mean. C is supposed to be 51 zero bytes > (408 > bits). Why is C bound to the number of bits. > > Bob > > On 09/03/2011 14:29, Vlad Dumitrescu wrote: > > Hi! > > > > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery > > > >> wrote: > > > > Hi > > > > I'm clearly not understanding the bit syntax properly. Could > someone > > explain to me what I am doing wrong here. Why does the first > not match > > when its the right shape but the second does when its the > wrong shape. > > The bound values don't make much sense to me either. > > > > 1> <> = > <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > > ** exception error: no match of right hand side value > > > > > > There is an "extra" 16#00 on the right side that is not matched > on the > > left. > > > > > > 2> <> = > <<16#effe02:24,16#00:48,16#00,(8*51):408>>. > > > > > > The larger size for A "swallows" the extra value, but shifts the > > values a byte to the right. > > > > regards, > > Vlad > > > > From bob@REDACTED Wed Mar 9 16:13:55 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 09 Mar 2011 15:13:55 +0000 Subject: [erlang-questions] Bit syntax In-Reply-To: <4D779847.3030601@bobcowdery.plus.com> References: <4D778CDF.3090907@bobcowdery.plus.com> <4D7793C6.7080903@bobcowdery.plus.com> <4D779847.3030601@bobcowdery.plus.com> Message-ID: <4D779933.7060602@bobcowdery.plus.com> Sorry, ignore the last message. same error, different reason because I forgot to f(). Bob On 09/03/2011 15:09, Bob Cowdery wrote: > What I actually want is 51 bytes of 16#00. > > 12> <> = <<16#effe02:24,16#00:48,16#00:408>>. > ** exception error: no match of right hand side value > > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > > Why is this different. Isn't it just saying 00 as a 408 bit number > rather than (8*51)? > > Bob > > On 09/03/2011 14:55, Vlad Dumitrescu wrote: >> Your C is 8*51 as a 408 bit number. You probably want 16#00:408 instead. >> >> /Vlad >> >> On Wed, Mar 9, 2011 at 15:50, Bob Cowdery > > wrote: >> >> Thanks. The syntax did look a bit odd for a repeating group. I >> pulled it >> off the web somewhere. >> >> So if I say: >> >> 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> >> . >> <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, >> 0,0,0,...>> >> 8> A. >> 15728130 >> 9> B. >> 0 >> 10> C. >> 408 >> >> What do these bound values mean. C is supposed to be 51 zero bytes >> (408 >> bits). Why is C bound to the number of bits. >> >> Bob >> >> On 09/03/2011 14:29, Vlad Dumitrescu wrote: >> > Hi! >> > >> > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery >> >> > > >> wrote: >> > >> > Hi >> > >> > I'm clearly not understanding the bit syntax properly. Could >> someone >> > explain to me what I am doing wrong here. Why does the first >> not match >> > when its the right shape but the second does when its the >> wrong shape. >> > The bound values don't make much sense to me either. >> > >> > 1> <> = >> <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >> > ** exception error: no match of right hand side value >> > >> > >> > There is an "extra" 16#00 on the right side that is not matched >> on the >> > left. >> > >> > >> > 2> <> = >> <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >> > >> > >> > The larger size for A "swallows" the extra value, but shifts the >> > values a byte to the right. >> > >> > regards, >> > Vlad >> > >> >> > From bob@REDACTED Wed Mar 9 16:17:34 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 9 Mar 2011 07:17:34 -0800 Subject: [erlang-questions] Bit syntax In-Reply-To: <4D779847.3030601@bobcowdery.plus.com> References: <4D778CDF.3090907@bobcowdery.plus.com> <4D7793C6.7080903@bobcowdery.plus.com> <4D779847.3030601@bobcowdery.plus.com> Message-ID: 8*51 is 408 (which is by coincidence the same as the number of bits), 16#00 is 0. Either way you're extending it to 408 bits. You're getting a bad match presumably because A, B, and/or C are already bound in your shell. Do an f(). first to forget all of the bound variables in the shell, and then it will work. 1> <> = <<16#effe02:24,16#00:48,16#00:408>>. <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,...>> 2> {A, B, C}. {15728130,0,0} On Wed, Mar 9, 2011 at 7:09 AM, Bob Cowdery wrote: > What I actually want is 51 bytes of 16#00. > > 12> <> = <<16#effe02:24,16#00:48,16#00:408>>. > ** exception error: no match of right hand side value > > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > ? ? ? ? ? ? ? ? ? ? ?0,0,0,...>> > > Why is this different. Isn't it just saying 00 as a 408 bit number > rather than (8*51)? > > Bob > > On 09/03/2011 14:55, Vlad Dumitrescu wrote: >> Your C is 8*51 as a 408 bit number. You probably want 16#00:408 instead. >> >> /Vlad >> >> On Wed, Mar 9, 2011 at 15:50, Bob Cowdery > > wrote: >> >> ? ? Thanks. The syntax did look a bit odd for a repeating group. I >> ? ? pulled it >> ? ? off the web somewhere. >> >> ? ? So if I say: >> >> ? ? 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> >> ? ? . >> ? ? <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, >> ? ? ?0,0,0,...>> >> ? ? 8> A. >> ? ? 15728130 >> ? ? 9> B. >> ? ? 0 >> ? ? 10> C. >> ? ? 408 >> >> ? ? What do these bound values mean. C is supposed to be 51 zero bytes >> ? ? (408 >> ? ? bits). Why is C bound to the number of bits. >> >> ? ? Bob >> >> ? ? On 09/03/2011 14:29, Vlad Dumitrescu wrote: >> ? ? > Hi! >> ? ? > >> ? ? > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery >> ? ? >> ? ? > > ? ? >> wrote: >> ? ? > >> ? ? > ? ? Hi >> ? ? > >> ? ? > ? ? I'm clearly not understanding the bit syntax properly. Could >> ? ? someone >> ? ? > ? ? explain to me what I am doing wrong here. Why does the first >> ? ? not match >> ? ? > ? ? when its the right shape but the second does when its the >> ? ? wrong shape. >> ? ? > ? ? The bound values don't make much sense to me either. >> ? ? > >> ? ? > ? ? 1> <> = >> ? ? <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >> ? ? > ? ? ** exception error: no match of right hand side value >> ? ? > >> ? ? > >> ? ? > There is an "extra" 16#00 on the right side that is not matched >> ? ? on the >> ? ? > left. >> ? ? > >> ? ? > >> ? ? > ? ? 2> <> = >> ? ? <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >> ? ? > >> ? ? > >> ? ? > The larger size for A "swallows" the extra value, but shifts the >> ? ? > values a byte to the right. >> ? ? > >> ? ? > regards, >> ? ? > Vlad >> ? ? > >> >> > > From bob@REDACTED Wed Mar 9 16:29:43 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 09 Mar 2011 15:29:43 +0000 Subject: [erlang-questions] Bit syntax In-Reply-To: References: <4D778CDF.3090907@bobcowdery.plus.com> <4D7793C6.7080903@bobcowdery.plus.com> <4D779847.3030601@bobcowdery.plus.com> Message-ID: <4D779CE7.4020409@bobcowdery.plus.com> Yes. I realised that as soon as I sent the message. I guess bad match accounts for a large number of error messages. All seems to be working now. Bob On 09/03/2011 15:17, Bob Ippolito wrote: > 8*51 is 408 (which is by coincidence the same as the number of bits), > 16#00 is 0. Either way you're extending it to 408 bits. > > You're getting a bad match presumably because A, B, and/or C are > already bound in your shell. Do an f(). first to forget all of the > bound variables in the shell, and then it will work. > > 1> <> = <<16#effe02:24,16#00:48,16#00:408>>. > <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,...>> > 2> {A, B, C}. > {15728130,0,0} > > On Wed, Mar 9, 2011 at 7:09 AM, Bob Cowdery wrote: >> What I actually want is 51 bytes of 16#00. >> >> 12> <> = <<16#effe02:24,16#00:48,16#00:408>>. >> ** exception error: no match of right hand side value >> >> <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, >> 0,0,0,...>> >> >> Why is this different. Isn't it just saying 00 as a 408 bit number >> rather than (8*51)? >> >> Bob >> >> On 09/03/2011 14:55, Vlad Dumitrescu wrote: >>> Your C is 8*51 as a 408 bit number. You probably want 16#00:408 instead. >>> >>> /Vlad >>> >>> On Wed, Mar 9, 2011 at 15:50, Bob Cowdery >> > wrote: >>> >>> Thanks. The syntax did look a bit odd for a repeating group. I >>> pulled it >>> off the web somewhere. >>> >>> So if I say: >>> >>> 7> <> = <<16#effe02:24,16#00:48,(8*51):408>> >>> . >>> <<239,254,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, >>> 0,0,0,...>> >>> 8> A. >>> 15728130 >>> 9> B. >>> 0 >>> 10> C. >>> 408 >>> >>> What do these bound values mean. C is supposed to be 51 zero bytes >>> (408 >>> bits). Why is C bound to the number of bits. >>> >>> Bob >>> >>> On 09/03/2011 14:29, Vlad Dumitrescu wrote: >>> > Hi! >>> > >>> > On Wed, Mar 9, 2011 at 15:21, Bob Cowdery >>> >>> > >> >> wrote: >>> > >>> > Hi >>> > >>> > I'm clearly not understanding the bit syntax properly. Could >>> someone >>> > explain to me what I am doing wrong here. Why does the first >>> not match >>> > when its the right shape but the second does when its the >>> wrong shape. >>> > The bound values don't make much sense to me either. >>> > >>> > 1> <> = >>> <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >>> > ** exception error: no match of right hand side value >>> > >>> > >>> > There is an "extra" 16#00 on the right side that is not matched >>> on the >>> > left. >>> > >>> > >>> > 2> <> = >>> <<16#effe02:24,16#00:48,16#00,(8*51):408>>. >>> > >>> > >>> > The larger size for A "swallows" the extra value, but shifts the >>> > values a byte to the right. >>> > >>> > regards, >>> > Vlad >>> > >>> >>> >> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From watson.timothy@REDACTED Wed Mar 9 17:53:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 9 Mar 2011 16:53:07 +0000 Subject: R14 vs R13 binary handling in driver_outputv Message-ID: Hi all, I have a linked in driver which receives data from port_command/2 in driver_outputv and casts the binary into some shape using a couple of structs. This approach works fine for R13, but in R14 I get extra data at the end of the binary. I'm guessing I've screwed up by making an assumption about the length of the binary - hoping someone can give me a nudge in the right direction. The iolist that goes to port_command/2 looks like this: DataSizeMarkers = [ size(Input), size(Xsl) ], [[ InputType, XslType, <<0:16/native-integer>> ], [ <> || Marker <- DataSizeMarkers ], [ Input, Xsl ]]; The variables are all binaries. This goes direct to driver outputv, where it is processed like so: ErlDrvBinary *bin = ev->binv[1]; int bin_size = bin->orig_size; Int32 offset = (Int32)(sizeof(InputSpecHeaders) + sizeof(PayloadSizeHeaders)); // error checking omitted for brevity const char *buffer = &(bin->orig_bytes[0]); const InputSpecHeaders *hspec = (const InputSpecHeaders* const)buffer; const PayloadSizeHeaders *hsoffset = (const PayloadSizeHeaders*)(hspec + 1); PayloadSizeHeaders hsize = *hsoffset; hsoffset++; buffer = (const char*)hsoffset; And those structures: typedef int8_t Int8; typedef int16_t Int16; typedef int32_t Int32; typedef struct { Int8 input_kind; Int8 xsl_kind; Int16 param_grp_arity; } InputSpecHeaders; typedef struct { Int32 input_size; Int32 xsl_size; } PayloadSizeHeaders; Source code is here: http://github.com/hyperthunk/erlxsl. Any advice would be most welcome - I'm still finding my way around linked-in drivers! Cheers, Tim From zvi.avraham@REDACTED Wed Mar 9 19:24:16 2011 From: zvi.avraham@REDACTED (Zvi) Date: Wed, 9 Mar 2011 10:24:16 -0800 (PST) Subject: Erlang Win32com In-Reply-To: References: Message-ID: <16cb1d7b-9116-4df8-8c3f-ce02ca570ba0@k38g2000vbn.googlegroups.com> Hi, There was old experimental COM binding for Erlang [1]. I think it's not supported anymore and removed from OTP distribution. I had idea to run it in old Erlang Win32 node. I didn't used it in the end - end up writing port in VBScript. [1] COMET - an Erlang-to-COM driver ftp.sunet.se/pub/lang/erlang/euc/00/Comet.ppt Zvi On Mar 2, 12:46?am, Michael Mackus wrote: > Is there a way to access win32com using erlang? > > For instance, in python, there's the "win32com" module. > > If you don't understand, I need a way to access functions in a windows DLL > from erlang. This is for a bot project I'm doing if you are curious, seems > like it'll get my feet wet with using Erlang. From oribrost@REDACTED Wed Mar 9 19:32:14 2011 From: oribrost@REDACTED (ori brost) Date: Wed, 9 Mar 2011 20:32:14 +0200 Subject: Making an Ubuntu daemon out of a release while using heart Message-ID: I have a release of a server program in Erlang that I would like to turn into a daemon. Currently I did this by writing a script (based on the release management script generated by rebar) which runs the Erlang VM synchronously, and writing an upstart .conf file which runs the script). My problem is that I want to use erl -heart. I have noticed that the whenever the VM dies, the heart process restarts it, but then the VM starts a new heart process and kills the old one. This means I have no permanent PIDs to follow. I was thinking of implementing my solution by having the script run a while-loop that will only exit if there is no heart process in the system (for now, assume there is no other erlang program running on the machine), and have upstart think that the script is the daemon. Does anyone have recommendation for a better solution? Or some general recommendation on how to make a daemon out of an Erlang server? Preferably using upstart? Best Regards, Ori Bar, Software Engineer Nivertech Ltd email: ori.bar@REDACTED From watson.timothy@REDACTED Wed Mar 9 20:59:16 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 9 Mar 2011 19:59:16 +0000 Subject: [erlang-questions] Re: Erlang Win32com In-Reply-To: <16cb1d7b-9116-4df8-8c3f-ce02ca570ba0@k38g2000vbn.googlegroups.com> References: <16cb1d7b-9116-4df8-8c3f-ce02ca570ba0@k38g2000vbn.googlegroups.com> Message-ID: On 9 March 2011 18:24, Zvi wrote: > [1] COMET - an Erlang-to-COM driver > ftp.sunet.se/pub/lang/erlang/euc/00/Comet.ppt > Never seen that one before - interesting. > On Mar 2, 12:46?am, Michael Mackus wrote: >> Is there a way to access win32com using erlang? Is it win32 or COM that you want? Do you just want to access win32 DLL with appropriate calling conventions and so on (in which case a NIF or linked-in driver would probably do it, along with a LoadLibrary system call) or do you actually want to create COM objects and use COM interfaces and such like? From antoine.koener@REDACTED Wed Mar 9 23:53:50 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Wed, 9 Mar 2011 23:53:50 +0100 Subject: [erlang-questions] RE: R14 vs R13 binary handling in driver_outputv In-Reply-To: References: Message-ID: <0711D4C9-8E9B-4DAD-AB96-1A26F608A965@gmail.com> Not a real answer but it's more about secure programming: why do you use 'signed' integer for offset and size ? Is offset going to be negative or even size ? On Mar 9, 2011, at 17:53 , Tim Watson wrote: > Hi all, > > I have a linked in driver which receives data from port_command/2 in > driver_outputv and casts the binary into some shape using a couple of > structs. This approach works fine for R13, but in R14 I get extra data > at the end of the binary. I'm guessing I've screwed up by making an > assumption about the length of the binary - hoping someone can give me > a nudge in the right direction. > > The iolist that goes to port_command/2 looks like this: > > DataSizeMarkers = [ size(Input), size(Xsl) ], > [[ InputType, XslType, <<0:16/native-integer>> ], > [ <> || Marker <- DataSizeMarkers ], > [ Input, Xsl ]]; > > The variables are all binaries. This goes direct to driver outputv, > where it is processed like so: > > ErlDrvBinary *bin = ev->binv[1]; > int bin_size = bin->orig_size; > Int32 offset = (Int32)(sizeof(InputSpecHeaders) + > sizeof(PayloadSizeHeaders)); > // error checking omitted for brevity > const char *buffer = &(bin->orig_bytes[0]); > const InputSpecHeaders *hspec = (const InputSpecHeaders* > const)buffer; > const PayloadSizeHeaders *hsoffset = (const PayloadSizeHeaders*) > (hspec + 1); > PayloadSizeHeaders hsize = *hsoffset; > hsoffset++; > buffer = (const char*)hsoffset; > > And those structures: > > typedef int8_t Int8; > typedef int16_t Int16; > typedef int32_t Int32; > > typedef struct { > Int8 input_kind; > Int8 xsl_kind; > Int16 param_grp_arity; > } InputSpecHeaders; > > typedef struct { > Int32 input_size; > Int32 xsl_size; > } PayloadSizeHeaders; > > > Source code is here: http://github.com/hyperthunk/erlxsl. > > Any advice would be most welcome - I'm still finding my way around > linked-in drivers! > > Cheers, > > Tim > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From mapandfold@REDACTED Thu Mar 10 09:11:57 2011 From: mapandfold@REDACTED (Justus) Date: Thu, 10 Mar 2011 16:11:57 +0800 Subject: dialyzer on re module (R14B01) Message-ID: Hi all, I think there is something missing in options (I think at least "unicode" is missing) of re module -spec, caused dialyzer say "Function filelib:fold_files/5 has no local return", which will further cause other function calling filelib:fold_files/5 be warned by dialyzer. -- Best Regards, Justus From watson.timothy@REDACTED Thu Mar 10 11:14:51 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 10 Mar 2011 10:14:51 +0000 Subject: [erlang-questions] RE: R14 vs R13 binary handling in driver_outputv In-Reply-To: <0711D4C9-8E9B-4DAD-AB96-1A26F608A965@gmail.com> References: <0711D4C9-8E9B-4DAD-AB96-1A26F608A965@gmail.com> Message-ID: On 9 March 2011 22:53, Koener Antoine wrote: > > Not a real answer but it's more about secure programming: > why do you use 'signed' integer for offset and size ? > > Is offset going to be negative or even size ? Thanks for pointing it out - I hadn't thought about it properly. Fixing it isn't going to fix my problem with R14 - clearly this isn't the right way to share a binary between the driver and the runtime. Still I'll go and refactor the code to use unsigned numbers while I'm waiting. From watson.timothy@REDACTED Thu Mar 10 11:44:56 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 10 Mar 2011 10:44:56 +0000 Subject: [erlang-questions] RE: R14 vs R13 binary handling in driver_outputv In-Reply-To: References: <0711D4C9-8E9B-4DAD-AB96-1A26F608A965@gmail.com> Message-ID: On 10 March 2011 10:14, Tim Watson wrote: > On 9 March 2011 22:53, Koener Antoine wrote: >> >> Not a real answer but it's more about secure programming: >> why do you use 'signed' integer for offset and size ? >> >> Is offset going to be negative or even size ? > > Thanks for pointing it out - I hadn't thought about it properly. > Fixing it isn't going to fix my problem with R14 - clearly this isn't > the right way to share a binary between the driver and the runtime. > Still I'll go and refactor the code to use unsigned numbers while I'm > waiting. > Fixed. Now can anyone suggest a better way of consuming the binary in driver_outputv? From gnoblin@REDACTED Thu Mar 10 13:34:57 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Thu, 10 Mar 2011 14:34:57 +0200 Subject: [erlang-questions] Re: Erlang: Who uses it for games? In-Reply-To: References: <25361468.416191297207444693.JavaMail.root@zimbra> <4D51D30F.4010108@gmail.com> Message-ID: wow (made with Erlang) http://www.pikkotekk.com/tech.html found it here http://www.unitypark3d.com/ (related to Unity3d game engine) From rob@REDACTED Thu Mar 10 16:02:43 2011 From: rob@REDACTED (Rob Harrop) Date: Thu, 10 Mar 2011 15:02:43 +0000 Subject: Mnesia Query Performance Message-ID: <4D78E813.5090003@rabbitmq.com> Hi, I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm a bit puzzled by the fact that read performance in transactions appears to be quadratic (or worse). I ran a test in which I do only reads (no writes, no matching) on both set and ordered_set tables. From the results, it is clear that transaction execution time does not increase linearly: [{ordered_set,500,11}, {ordered_set,1000,34}, {ordered_set,2000,120}, {ordered_set,4000,418}, {ordered_set,8000,1800}, {ordered_set,16000,9981}, {set,500,11}, {set,1000,35}, {set,2000,115}, {set,4000,406}, {set,8000,1808}, {set,16000,9804}] Here the second slot in each tuple is the number of reads and the third slot is tx execution time in milliseconds. I have attached the test code. Am I doing something horribly wrong, or is this expected behaviour? If this is expected behaviour are there are known/accepted workarounds? Kind Regards, Rob Harrop RabbitMQ -------------- next part -------------- A non-text attachment was scrubbed... Name: mtest.erl Type: text/x-erlang Size: 1059 bytes Desc: not available URL: From mevans@REDACTED Thu Mar 10 16:25:55 2011 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 10 Mar 2011 10:25:55 -0500 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: <4D78E813.5090003@rabbitmq.com> References: <4D78E813.5090003@rabbitmq.com> Message-ID: Hi, I changed your read function to look like this instead: read([]) -> ok; read([Key|Keys]) -> {atomic, _} = tx(fun() -> mnesia:read(test_record, Key) end), read(Keys). Before the change my test mirrored yours: [{ordered_set,500,17}, {ordered_set,1000,43}, {ordered_set,2000,139}, {ordered_set,4000,492}, {ordered_set,8000,1882}, {ordered_set,16000,7156}, {set,500,14}, {set,1000,49}, {set,2000,140}, {set,4000,493}, {set,8000,1846}, {set,16000,7123}] After the change it looks like: [{ordered_set,500,28}, {ordered_set,1000,41}, {ordered_set,2000,82}, {ordered_set,4000,166}, {ordered_set,8000,335}, {ordered_set,16000,689}, {set,500,20}, {set,1000,41}, {set,2000,84}, {set,4000,167}, {set,8000,337}, {set,16000,701}] I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop Sent: Thursday, March 10, 2011 10:03 AM To: erlang-questions@REDACTED Subject: [erlang-questions] Mnesia Query Performance Hi, I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm a bit puzzled by the fact that read performance in transactions appears to be quadratic (or worse). I ran a test in which I do only reads (no writes, no matching) on both set and ordered_set tables. From the results, it is clear that transaction execution time does not increase linearly: [{ordered_set,500,11}, {ordered_set,1000,34}, {ordered_set,2000,120}, {ordered_set,4000,418}, {ordered_set,8000,1800}, {ordered_set,16000,9981}, {set,500,11}, {set,1000,35}, {set,2000,115}, {set,4000,406}, {set,8000,1808}, {set,16000,9804}] Here the second slot in each tuple is the number of reads and the third slot is tx execution time in milliseconds. I have attached the test code. Am I doing something horribly wrong, or is this expected behaviour? If this is expected behaviour are there are known/accepted workarounds? Kind Regards, Rob Harrop RabbitMQ From rob@REDACTED Thu Mar 10 17:55:00 2011 From: rob@REDACTED (Rob Harrop) Date: Thu, 10 Mar 2011 16:55:00 +0000 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: References: <4D78E813.5090003@rabbitmq.com> Message-ID: <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> Hi Matt, Thanks for taking the time to look at this. I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction. Is this the recommended solution? It's a real shame to lose the transaction protection here. Regards, Rob On 10 Mar 2011, at 15:25, "Evans, Matthew" wrote: > Hi, > > I changed your read function to look like this instead: > > read([]) -> > ok; > read([Key|Keys]) -> > {atomic, _} = > tx(fun() -> > mnesia:read(test_record, Key) > end), > read(Keys). > > Before the change my test mirrored yours: > > [{ordered_set,500,17}, > {ordered_set,1000,43}, > {ordered_set,2000,139}, > {ordered_set,4000,492}, > {ordered_set,8000,1882}, > {ordered_set,16000,7156}, > {set,500,14}, > {set,1000,49}, > {set,2000,140}, > {set,4000,493}, > {set,8000,1846}, > {set,16000,7123}] > > > After the change it looks like: > > [{ordered_set,500,28}, > {ordered_set,1000,41}, > {ordered_set,2000,82}, > {ordered_set,4000,166}, > {ordered_set,8000,335}, > {ordered_set,16000,689}, > {set,500,20}, > {set,1000,41}, > {set,2000,84}, > {set,4000,167}, > {set,8000,337}, > {set,16000,701}] > > I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. > > Matt > > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop > Sent: Thursday, March 10, 2011 10:03 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions] Mnesia Query Performance > > Hi, > > I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm > a bit puzzled by the fact that read performance in transactions appears > to be quadratic (or worse). > > I ran a test in which I do only reads (no writes, no matching) on both > set and ordered_set tables. From the results, it is clear that > transaction execution time does not increase linearly: > > [{ordered_set,500,11}, > {ordered_set,1000,34}, > {ordered_set,2000,120}, > {ordered_set,4000,418}, > {ordered_set,8000,1800}, > {ordered_set,16000,9981}, > {set,500,11}, > {set,1000,35}, > {set,2000,115}, > {set,4000,406}, > {set,8000,1808}, > {set,16000,9804}] > > Here the second slot in each tuple is the number of reads and the third > slot is tx execution time in milliseconds. > > I have attached the test code. > > Am I doing something horribly wrong, or is this expected behaviour? If > this is expected behaviour are there are known/accepted workarounds? > > Kind Regards, > > Rob Harrop > RabbitMQ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From dgud@REDACTED Thu Mar 10 18:56:45 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Thu, 10 Mar 2011 18:56:45 +0100 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> References: <4D78E813.5090003@rabbitmq.com> <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> Message-ID: What happens if you grab a table lock the first thing in the transaction? /Dan On Thu, Mar 10, 2011 at 5:55 PM, Rob Harrop wrote: > Hi Matt, > > Thanks for taking the time to look at this. > > I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction. > > Is this the recommended solution? It's a real shame to lose the transaction protection here. > > Regards, > > Rob > > > > On 10 Mar 2011, at 15:25, "Evans, Matthew" wrote: > >> Hi, >> >> I changed your read function to look like this instead: >> >> read([]) -> >> ? ?ok; >> read([Key|Keys]) -> >> ? ?{atomic, _} = >> ? ? ? ?tx(fun() -> >> ? ? ? ? ? ? ? ? ? mnesia:read(test_record, Key) >> ? ? ? ? ? end), >> ? ?read(Keys). >> >> Before the change my test mirrored yours: >> >> [{ordered_set,500,17}, >> {ordered_set,1000,43}, >> {ordered_set,2000,139}, >> {ordered_set,4000,492}, >> {ordered_set,8000,1882}, >> {ordered_set,16000,7156}, >> {set,500,14}, >> {set,1000,49}, >> {set,2000,140}, >> {set,4000,493}, >> {set,8000,1846}, >> {set,16000,7123}] >> >> >> After the change it looks like: >> >> [{ordered_set,500,28}, >> {ordered_set,1000,41}, >> {ordered_set,2000,82}, >> {ordered_set,4000,166}, >> {ordered_set,8000,335}, >> {ordered_set,16000,689}, >> {set,500,20}, >> {set,1000,41}, >> {set,2000,84}, >> {set,4000,167}, >> {set,8000,337}, >> {set,16000,701}] >> >> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. >> >> Matt >> >> >> -----Original Message----- >> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop >> Sent: Thursday, March 10, 2011 10:03 AM >> To: erlang-questions@REDACTED >> Subject: [erlang-questions] Mnesia Query Performance >> >> Hi, >> >> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm >> a bit puzzled by the fact that read performance in transactions appears >> to be quadratic (or worse). >> >> I ran a test in which I do only reads (no writes, no matching) on both >> set and ordered_set tables. From the results, it is clear that >> transaction execution time does not increase linearly: >> >> [{ordered_set,500,11}, >> ?{ordered_set,1000,34}, >> ?{ordered_set,2000,120}, >> ?{ordered_set,4000,418}, >> ?{ordered_set,8000,1800}, >> ?{ordered_set,16000,9981}, >> ?{set,500,11}, >> ?{set,1000,35}, >> ?{set,2000,115}, >> ?{set,4000,406}, >> ?{set,8000,1808}, >> ?{set,16000,9804}] >> >> Here the second slot in each tuple is the number of reads and the third >> slot is tx execution time in milliseconds. >> >> I have attached the test code. >> >> Am I doing something horribly wrong, or is this expected behaviour? If >> this is expected behaviour are there are known/accepted workarounds? >> >> Kind Regards, >> >> Rob Harrop >> RabbitMQ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rob@REDACTED Thu Mar 10 19:07:49 2011 From: rob@REDACTED (Rob Harrop) Date: Thu, 10 Mar 2011 18:07:49 +0000 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: References: <4D78E813.5090003@rabbitmq.com> <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> Message-ID: <4D791375.5040407@rabbitmq.com> That makes a big difference - performance is linear when the transaction takes a read lock on the table first. Thanks for that, Rob On 10/03/11 17:56, Dan Gudmundsson wrote: > What happens if you grab a table lock the first thing in the transaction? > > /Dan > > On Thu, Mar 10, 2011 at 5:55 PM, Rob Harrop wrote: >> Hi Matt, >> >> Thanks for taking the time to look at this. >> >> I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction. >> >> Is this the recommended solution? It's a real shame to lose the transaction protection here. >> >> Regards, >> >> Rob >> >> >> >> On 10 Mar 2011, at 15:25, "Evans, Matthew" wrote: >> >>> Hi, >>> >>> I changed your read function to look like this instead: >>> >>> read([]) -> >>> ok; >>> read([Key|Keys]) -> >>> {atomic, _} = >>> tx(fun() -> >>> mnesia:read(test_record, Key) >>> end), >>> read(Keys). >>> >>> Before the change my test mirrored yours: >>> >>> [{ordered_set,500,17}, >>> {ordered_set,1000,43}, >>> {ordered_set,2000,139}, >>> {ordered_set,4000,492}, >>> {ordered_set,8000,1882}, >>> {ordered_set,16000,7156}, >>> {set,500,14}, >>> {set,1000,49}, >>> {set,2000,140}, >>> {set,4000,493}, >>> {set,8000,1846}, >>> {set,16000,7123}] >>> >>> >>> After the change it looks like: >>> >>> [{ordered_set,500,28}, >>> {ordered_set,1000,41}, >>> {ordered_set,2000,82}, >>> {ordered_set,4000,166}, >>> {ordered_set,8000,335}, >>> {ordered_set,16000,689}, >>> {set,500,20}, >>> {set,1000,41}, >>> {set,2000,84}, >>> {set,4000,167}, >>> {set,8000,337}, >>> {set,16000,701}] >>> >>> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. >>> >>> Matt >>> >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop >>> Sent: Thursday, March 10, 2011 10:03 AM >>> To: erlang-questions@REDACTED >>> Subject: [erlang-questions] Mnesia Query Performance >>> >>> Hi, >>> >>> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm >>> a bit puzzled by the fact that read performance in transactions appears >>> to be quadratic (or worse). >>> >>> I ran a test in which I do only reads (no writes, no matching) on both >>> set and ordered_set tables. From the results, it is clear that >>> transaction execution time does not increase linearly: >>> >>> [{ordered_set,500,11}, >>> {ordered_set,1000,34}, >>> {ordered_set,2000,120}, >>> {ordered_set,4000,418}, >>> {ordered_set,8000,1800}, >>> {ordered_set,16000,9981}, >>> {set,500,11}, >>> {set,1000,35}, >>> {set,2000,115}, >>> {set,4000,406}, >>> {set,8000,1808}, >>> {set,16000,9804}] >>> >>> Here the second slot in each tuple is the number of reads and the third >>> slot is tx execution time in milliseconds. >>> >>> I have attached the test code. >>> >>> Am I doing something horribly wrong, or is this expected behaviour? If >>> this is expected behaviour are there are known/accepted workarounds? >>> >>> Kind Regards, >>> >>> Rob Harrop >>> RabbitMQ >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From mevans@REDACTED Thu Mar 10 20:34:15 2011 From: mevans@REDACTED (Evans, Matthew) Date: Thu, 10 Mar 2011 14:34:15 -0500 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: <4D791375.5040407@rabbitmq.com> References: <4D78E813.5090003@rabbitmq.com> <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> <4D791375.5040407@rabbitmq.com> Message-ID: That's interesting. I wonder why that would be? By that what is the difference between getting a lock first in a transaction, then performing the mnesia:read within a list comprehension in that transaction verses separate transactions for each read? I would've thought the two would be the same.... Matt -----Original Message----- From: Rob Harrop [mailto:rob@REDACTED] Sent: Thursday, March 10, 2011 1:08 PM To: Dan Gudmundsson Cc: Evans, Matthew; erlang-questions@REDACTED Subject: Re: [erlang-questions] Mnesia Query Performance That makes a big difference - performance is linear when the transaction takes a read lock on the table first. Thanks for that, Rob On 10/03/11 17:56, Dan Gudmundsson wrote: > What happens if you grab a table lock the first thing in the transaction? > > /Dan > > On Thu, Mar 10, 2011 at 5:55 PM, Rob Harrop wrote: >> Hi Matt, >> >> Thanks for taking the time to look at this. >> >> I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction. >> >> Is this the recommended solution? It's a real shame to lose the transaction protection here. >> >> Regards, >> >> Rob >> >> >> >> On 10 Mar 2011, at 15:25, "Evans, Matthew" wrote: >> >>> Hi, >>> >>> I changed your read function to look like this instead: >>> >>> read([]) -> >>> ok; >>> read([Key|Keys]) -> >>> {atomic, _} = >>> tx(fun() -> >>> mnesia:read(test_record, Key) >>> end), >>> read(Keys). >>> >>> Before the change my test mirrored yours: >>> >>> [{ordered_set,500,17}, >>> {ordered_set,1000,43}, >>> {ordered_set,2000,139}, >>> {ordered_set,4000,492}, >>> {ordered_set,8000,1882}, >>> {ordered_set,16000,7156}, >>> {set,500,14}, >>> {set,1000,49}, >>> {set,2000,140}, >>> {set,4000,493}, >>> {set,8000,1846}, >>> {set,16000,7123}] >>> >>> >>> After the change it looks like: >>> >>> [{ordered_set,500,28}, >>> {ordered_set,1000,41}, >>> {ordered_set,2000,82}, >>> {ordered_set,4000,166}, >>> {ordered_set,8000,335}, >>> {ordered_set,16000,689}, >>> {set,500,20}, >>> {set,1000,41}, >>> {set,2000,84}, >>> {set,4000,167}, >>> {set,8000,337}, >>> {set,16000,701}] >>> >>> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. >>> >>> Matt >>> >>> >>> -----Original Message----- >>> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop >>> Sent: Thursday, March 10, 2011 10:03 AM >>> To: erlang-questions@REDACTED >>> Subject: [erlang-questions] Mnesia Query Performance >>> >>> Hi, >>> >>> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm >>> a bit puzzled by the fact that read performance in transactions appears >>> to be quadratic (or worse). >>> >>> I ran a test in which I do only reads (no writes, no matching) on both >>> set and ordered_set tables. From the results, it is clear that >>> transaction execution time does not increase linearly: >>> >>> [{ordered_set,500,11}, >>> {ordered_set,1000,34}, >>> {ordered_set,2000,120}, >>> {ordered_set,4000,418}, >>> {ordered_set,8000,1800}, >>> {ordered_set,16000,9981}, >>> {set,500,11}, >>> {set,1000,35}, >>> {set,2000,115}, >>> {set,4000,406}, >>> {set,8000,1808}, >>> {set,16000,9804}] >>> >>> Here the second slot in each tuple is the number of reads and the third >>> slot is tx execution time in milliseconds. >>> >>> I have attached the test code. >>> >>> Am I doing something horribly wrong, or is this expected behaviour? If >>> this is expected behaviour are there are known/accepted workarounds? >>> >>> Kind Regards, >>> >>> Rob Harrop >>> RabbitMQ >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From dgud@REDACTED Thu Mar 10 21:33:50 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Thu, 10 Mar 2011 21:33:50 +0100 Subject: [erlang-questions] Mnesia Query Performance In-Reply-To: References: <4D78E813.5090003@rabbitmq.com> <220571F6-441A-4385-859A-EA7230325BE4@rabbitmq.com> <4D791375.5040407@rabbitmq.com> Message-ID: A separate transaction for each read will have much more overhead. Much of the overhead in the transaction is grabbing locks, if you grab a table lock first then each read (or write if write-locked) can skip taking a lock. The quadratic behaviour comes from the implementation, mnesia stores the grabbed locks in a ets bag table. Probably using the table as key, which causes the longer and longer insertion time for each lock that is taken on the same table*. This shouldn't be a "real" problem because if you access so many records from the same table you should grab a table lock instead of taking a lock per record. /Dan * I guess I don't have the code on my windows machine at home. On Thu, Mar 10, 2011 at 8:34 PM, Evans, Matthew wrote: > That's interesting. > > I wonder why that would be? By that what is the difference between getting a lock first in a transaction, then performing the mnesia:read within a list comprehension in that transaction verses separate transactions for each read? > > I would've thought the two would be the same.... > > Matt > > -----Original Message----- > From: Rob Harrop [mailto:rob@REDACTED] > Sent: Thursday, March 10, 2011 1:08 PM > To: Dan Gudmundsson > Cc: Evans, Matthew; erlang-questions@REDACTED > Subject: Re: [erlang-questions] Mnesia Query Performance > > That makes a big difference - performance is linear when the transaction > takes a read lock on the table first. > > Thanks for that, > > Rob > > On 10/03/11 17:56, Dan Gudmundsson wrote: >> What happens if you grab a table lock the first thing in the transaction? >> >> /Dan >> >> On Thu, Mar 10, 2011 at 5:55 PM, Rob Harrop ?wrote: >>> Hi Matt, >>> >>> Thanks for taking the time to look at this. >>> >>> I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction. >>> >>> Is this the recommended solution? It's a real shame to lose the transaction protection here. >>> >>> Regards, >>> >>> Rob >>> >>> >>> >>> On 10 Mar 2011, at 15:25, "Evans, Matthew" ?wrote: >>> >>>> Hi, >>>> >>>> I changed your read function to look like this instead: >>>> >>>> read([]) -> >>>> ? ? ok; >>>> read([Key|Keys]) -> >>>> ? ? {atomic, _} = >>>> ? ? ? ? tx(fun() -> >>>> ? ? ? ? ? ? ? ? ? ?mnesia:read(test_record, Key) >>>> ? ? ? ? ? ?end), >>>> ? ? read(Keys). >>>> >>>> Before the change my test mirrored yours: >>>> >>>> [{ordered_set,500,17}, >>>> {ordered_set,1000,43}, >>>> {ordered_set,2000,139}, >>>> {ordered_set,4000,492}, >>>> {ordered_set,8000,1882}, >>>> {ordered_set,16000,7156}, >>>> {set,500,14}, >>>> {set,1000,49}, >>>> {set,2000,140}, >>>> {set,4000,493}, >>>> {set,8000,1846}, >>>> {set,16000,7123}] >>>> >>>> >>>> After the change it looks like: >>>> >>>> [{ordered_set,500,28}, >>>> {ordered_set,1000,41}, >>>> {ordered_set,2000,82}, >>>> {ordered_set,4000,166}, >>>> {ordered_set,8000,335}, >>>> {ordered_set,16000,689}, >>>> {set,500,20}, >>>> {set,1000,41}, >>>> {set,2000,84}, >>>> {set,4000,167}, >>>> {set,8000,337}, >>>> {set,16000,701}] >>>> >>>> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why. >>>> >>>> Matt >>>> >>>> >>>> -----Original Message----- >>>> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop >>>> Sent: Thursday, March 10, 2011 10:03 AM >>>> To: erlang-questions@REDACTED >>>> Subject: [erlang-questions] Mnesia Query Performance >>>> >>>> Hi, >>>> >>>> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm >>>> a bit puzzled by the fact that read performance in transactions appears >>>> to be quadratic (or worse). >>>> >>>> I ran a test in which I do only reads (no writes, no matching) on both >>>> set and ordered_set tables. From the results, it is clear that >>>> transaction execution time does not increase linearly: >>>> >>>> [{ordered_set,500,11}, >>>> ? {ordered_set,1000,34}, >>>> ? {ordered_set,2000,120}, >>>> ? {ordered_set,4000,418}, >>>> ? {ordered_set,8000,1800}, >>>> ? {ordered_set,16000,9981}, >>>> ? {set,500,11}, >>>> ? {set,1000,35}, >>>> ? {set,2000,115}, >>>> ? {set,4000,406}, >>>> ? {set,8000,1808}, >>>> ? {set,16000,9804}] >>>> >>>> Here the second slot in each tuple is the number of reads and the third >>>> slot is tx execution time in milliseconds. >>>> >>>> I have attached the test code. >>>> >>>> Am I doing something horribly wrong, or is this expected behaviour? If >>>> this is expected behaviour are there are known/accepted workarounds? >>>> >>>> Kind Regards, >>>> >>>> Rob Harrop >>>> RabbitMQ >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kuleshovmail@REDACTED Fri Mar 11 06:41:42 2011 From: kuleshovmail@REDACTED (shk) Date: Thu, 10 Mar 2011 21:41:42 -0800 (PST) Subject: Escript to run erlang applcation Message-ID: <1299822102670-3347510.post@n4.nabble.com> Hello, I have erlang *.app and some *.erl files. I compiled it. Hoc can i write escript script to run erlang *.app? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Escript-to-run-erlang-applcation-tp3347510p3347510.html Sent from the Erlang Questions mailing list archive at Nabble.com. From bo.zhang86@REDACTED Fri Mar 11 07:40:14 2011 From: bo.zhang86@REDACTED (zhangbo) Date: Fri, 11 Mar 2011 14:40:14 +0800 Subject: about websocket 76 in erlang Message-ID: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> Hi, I've been writing a simple websocket(draft 76) server, but I caught a odd question. When open the html, the erlang console print "data Received:{tcp_closed,#Port<0.2108>}". But if I use jetty instead of erlang, it's ok. Please help. Thank you. Here is the code: Html: WebSoket Demo
erlang: -module(websocketd). -compile(export_all). -author({jha, abhinav}). start() -> {ok, L} = gen_tcp:listen(8000, [binary, {active, true}, {reuseaddr, true}, {packet, 0}]), erlang:process_flag(trap_exit, true), Pid = spawn_link(websocketd, acceptor, [L]), receive {'EXIT', Pid, Why} -> io:format("Process ~p exited with ~p. ~n", [Pid, Why]); Any -> io:format("~p~n", [Any]) end. % Accepts multiple connections and handles them separately. acceptor(L)-> {ok, S} = gen_tcp:accept(L), spawn(fun()->acceptor(L) end), handle(S). % Not really required, just a wrapper over handle/2 in case we want to do something later. handle(S)-> handle(S, []). handle(S, _Data)-> Pid = self(), Child = spawn(fun() -> handshake_and_talk(Pid) end), loop(S, Child). loop(S, Child)-> receive {tcp, S, Bin} -> Child ! {self(), Bin}, loop(S, Child); {Child, X} -> gen_tcp:send(S, X), loop(S, Child); _Any -> loop(S, Child) end. % Handshake with the client and begin talking. handshake_and_talk(Ppid)-> receive {Ppid, X} -> % Body = the checksum comprised after processing Key1, Key2 and the Client's request body. Body = process_client_handshake(binary_to_list(X)), % Send the Handshake stuff to the browser. Ppid ! {self(), "HTTP/1.1 101 WebSocket Protocol Handshake\r\n"}, Ppid ! {self(), "Upgrade: WebSocket\r\n"}, Ppid ! {self(), "Connection: Upgrade\r\n"}, Ppid ! {self(), "Sec-WebSocket-Origin: http://127.0.0.1:8080\r\n"}, Ppid ! {self(), "Sec-WebSocket-Location: ws://127.0.0.1:8000/\r\n"}, Ppid ! {self(), "Sec-WebSocket-Protocol: chat"}, Ppid ! {self(), "\r\n\r\n"}, % Send the body. Ppid ! {self(), Body}, % Now call the talk method to do the actual talking with the browser. talk(Ppid); Any -> io:format("[Child] Random stuff received:~p~n. ~p", [Any, Ppid]) end. % Function to actuall talk to the browser. % Dummy implementation - Implement as reuired. talk(Browser) -> receive after 1000 -> % This is the main communicator function to the Browser. % Whatever you write instead of "Hahah" will get sent to the browser. % the 0 and 255 is the framing ( required) - don't change that. Browser ! {self(), [0]}, Browser ! {self(), "Hahah"}, Browser ! {self(), [255]}, talk(Browser) end. % Process client's handshake to retrieve information. process_client_handshake(X)-> [Body|Head] = lists:reverse(string:tokens(X, "\r\n")), {Key1, Key2} = extract_keys(lists:reverse(Head)), {Skey1, Skey2} = process_keys(Key1, Key2), Bin_body = list_to_binary(Body), Key = <>, erlang:md5(Key). % Extract keys from the client's handshake. extract_keys([H|T])-> Key1 = extract_key("Sec-WebSocket-Key1: ", [H|T]), Key2 = extract_key("Sec-WebSocket-Key2: ", [H|T]), {Key1, Key2}. extract_key(X, [H|T])-> case string:str(H, X) of 0 -> extract_key(X, T); _Pos -> string:substr(H, string:len(X) + 1) end. % Process the keys as mentioned in the handshake 76 draft of the ietf. process_keys(Key1, Key2)-> {Digits1, []} = string:to_integer(digits(Key1)), {Digits2, []} = string:to_integer(digits(Key2)), Spaces1 = spaces(Key1), Spaces2 = spaces(Key2), {Digits1 div Spaces1, Digits2 div Spaces2}. % Concatenate digits 0-9 of a string digits(X)-> [A || A<-X, A =< 57, A >= 48]. % Count number of spaces in a string. spaces(X)-> string:len([ A || A<-X, A =:= 32]). Bob From elinsn@REDACTED Fri Mar 11 09:38:21 2011 From: elinsn@REDACTED (Sergey Yelin) Date: Fri, 11 Mar 2011 11:38:21 +0300 Subject: Pool and hosts with different users Message-ID: Hi list, playing with erlang poll a have found interesting feature. Access to remote hosts implicitly requires existing user with the same name in all hosts listed in .hosts.erlang file (because underlying transport, e.g. rsh/ssh requires it). In my case I have a few hosts with different user names, so I need explicitly specify user name for every hosts. According ssh rules I try to specify it in .hosts.erlang as "'user@REDACTED'." , it works in one direction only: master can login to slave host but can't start slave node because of wrong name format "queue@REDACTED@host". So, is there any other ways to specify user for slave nodes? File .hosts.erlang on master host (robby): 'slave@REDACTED'. Error on slave node (found in ssh debug log): slave@REDACTED:~$ erl -master pq@REDACTED -sname pq@REDACTED@debian -s slave slave_start pq@REDACTED slave_waiter_0 {error_logger,{{2011,3,2},{22,59,6}},"Invalid node name: ~p~n",['pq@REDACTED @debian']} {error_logger,{{2011,3,2},{22,59,6}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.20.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.9.0>]},{messages,[]},{links,[<0.17.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,377},{stack_size,24},{reductions,449}],[]]} {error_logger,{{2011,3,2},{22,59,6}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[['pq@REDACTED @debian',shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]} {error_logger,{{2011,3,2},{22,59,6}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,shutdown},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]} {error_logger,{{2011,3,2},{22,59,6}},std_info,[{application,kernel},{exited,{shutdown,{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}}"} From bob@REDACTED Fri Mar 11 10:01:38 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 11 Mar 2011 09:01:38 +0000 Subject: Binary manipulation Message-ID: <4D79E4F2.2010906@bobcowdery.plus.com> Hi I would appreciate any advice/ideas on how to efficiently handle a large binary. The binary in question has a couple of protocol wrappers around it which are no problem as the bit syntax easily copes with that. What I'm not sure of is how to handle the data. <> What I have to do is extract I and Q which are 24 bit le values and end up with another binary which is: <> Where I and Q are now floating point numbers. I will need M in the future, but not right now. It's a little more complicated than that because several of the raw data binaries will go into the processed binary which will be a multiple of 64 up to 1024. I will also need to do the opposite, i.e. take the processed binary and transform it back to the raw data binary. Thanks Bob From lionel.cons@REDACTED Fri Mar 11 12:38:16 2011 From: lionel.cons@REDACTED (Lionel Cons) Date: Fri, 11 Mar 2011 12:38:16 +0100 Subject: Idiomatic iterators Message-ID: <201103111138.p2BBcGbM024295@lxadm11.cern.ch> I'm porting some code to Erlang that provides access to a list of "elements". Since this list can potentially be big, I don't want to return it in one go. In the other supported languages, we use simple iterators (e.g. first() and next()). . What is the most idiomatic way to provide iterators in Erlang? So far, I thought about the following two approaches. (1) Provide a function that would call a user supplied function on each elements in the list. Something like: X = mymodule:new(), mymodule:iterate(X, fun (E) -> ... end), (2) Provide a function that creates an iterator and another one that returns one element at a time (as well as a new iterator to keep state). Something like: X = mymodule:new(), I0 = mymodule:iterator(X), {I1, E1} = mymodule:next(X, I0), {I2, E2} = mymodule:next(X, I1), % could end with Ex = undefined (1) is simpler but it is also more limited as you can only get one element at a time. (2) is more flexible and can easily be used in an Erlang-style "loop" but it requires more code to use. There are probably other ways to do it, what would be the most idiomatic? Thanks in advance for your advices. Lionel Cons From rob@REDACTED Fri Mar 11 12:49:10 2011 From: rob@REDACTED (Rob Harrop) Date: Fri, 11 Mar 2011 11:49:10 +0000 Subject: [erlang-questions] Idiomatic iterators In-Reply-To: <201103111138.p2BBcGbM024295@lxadm11.cern.ch> References: <201103111138.p2BBcGbM024295@lxadm11.cern.ch> Message-ID: <4D7A0C36.7090008@rabbitmq.com> Lionel, There aren't many places in the standard library where you have an iterator pattern like (2) - only gb_trees comes to mind. I prefer something like mymodule:foreach (to be consistent with lists). You might also want to consider adding a mymodule:fold operation which will allow you to do pretty much all the processing you want on your stream of elements. How big is your list likely to be? Is it something that could start taxing the memory of the Erlang process? Regards, Rob On 11/03/11 11:38, Lionel Cons wrote: > > > I'm porting some code to Erlang that provides access to a list of "elements". > > Since this list can potentially be big, I don't want to return it in one go. > > In the other supported languages, we use simple iterators (e.g. first() and next()). > > . > > What is the most idiomatic way to provide iterators in Erlang? > > So far, I thought about the following two approaches. > > (1) Provide a function that would call a user supplied function on each > elements in the list. Something like: > > X = mymodule:new(), > mymodule:iterate(X, fun (E) -> ... end), > > (2) Provide a function that creates an iterator and another one that returns > one element at a time (as well as a new iterator to keep state). Something > like: > > X = mymodule:new(), > I0 = mymodule:iterator(X), > {I1, E1} = mymodule:next(X, I0), > {I2, E2} = mymodule:next(X, I1), > % could end with Ex = undefined > > (1) is simpler but it is also more limited as you can only get one element > at a time. > > (2) is more flexible and can easily be used in an Erlang-style "loop" but it > requires more code to use. > > There are probably other ways to do it, what would be the most idiomatic? > > Thanks in advance for your advices. > > Lionel Cons > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From fred.hebert@REDACTED Fri Mar 11 13:12:38 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 11 Mar 2011 07:12:38 -0500 Subject: [erlang-questions] Idiomatic iterators In-Reply-To: <201103111138.p2BBcGbM024295@lxadm11.cern.ch> References: <201103111138.p2BBcGbM024295@lxadm11.cern.ch> Message-ID: There are two different reasons for iterators there. The first one is to avoid producing the elements in the first place, and the second one is to just provide an API to let you work on them one at a time. In the first case, you want to only fetch a few elements at a time: they might be in a file, a DB, or even just bits of information you can generate mathematically. In this case, you can look at the APIs of ETS and Mnesia, which use the form Mod:next/2 and Mod:prev/2 (http://erldocs.com/R14B01/mnesia/mnesia.html#next/2). This type of API is rather flexible because it allows random access to many readers based off the keys. If you're able to hold your data set in one of these databases, you'd benefit from iterators right there. If what you want is the iterator so you can work on each element one at a time, but don't really mind holding them all in memory at once you might be fine using something like the queue module and popping elements off the queue one at a time. Otherwise, there's the generalisation of that queue called a 'zipper'. I give a quick explanation on my blog (http://ferd.ca/yet-another-article-on-zippers.html) and have a little library with them (https://bitbucket.org/ferd/zippers/src) that can be installed with agner (http://erlagner.org) too. Zippers have the advantage of being bi-directional (you can go forward or backwards) and can insert at the current point you're in the structure, while the queues in the standard library allow you to push element on the queue and then remove them in O(1) amortised time. Last of all, you have gb_trees and gb_sets, which provide an iterator to the data structure that supports a single operation, next/1. gb_trees is used for key-value stores while sets are for sets (duh) of unique elements. These are mostly useful because they allow to amortise the conversion to a list over each element as opposed to at the beginning. -- Fred H?bert http://www.erlang-solutions.com From per.melin@REDACTED Fri Mar 11 15:05:36 2011 From: per.melin@REDACTED (Per Melin) Date: Fri, 11 Mar 2011 15:05:36 +0100 Subject: [erlang-questions] about websocket 76 in erlang In-Reply-To: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> Message-ID: On Fri, Mar 11, 2011 at 7:40 AM, zhangbo wrote: > ? ? ? ?I've been writing a simple websocket(draft 76) server, but I caught a odd question. When open the html, the erlang console print "data Received:{tcp_closed,#Port<0.2108>}". But if I use jetty instead of erlang, it's ok. Please help. Thank you. Works for me in Safari and Chrome (after I change the javascript to open port 8000 instead of 8080). > ? ? ? ? ? ?function display() { > ? ? ? ? ? ? ? ?var valueLabel = document.getElementById("valueLabel"); > ? ? ? ? ? ? ? ?valueLabel.innerHTML = ""; > ? ? ? ? ? ? ? ?var ws = new WebSocket("ws://127.0.0.1:8080"); > ? ? ? ? ? ? ? ?ws.onmessage = function(evt) { > ? ? ? ? ? ? ? ? ? ? ? ?alert("msg:" + evt.data); > ? ? ? ? ? ? ? ? ? ?valueLabel.innerHTML = evt.data; > ? ? ? ? ? ? ? ?}; > > ? ? ? ? ? ? ? ?ws.onclose = function() { > ? ? ? ? ? ? ? ?}; > > ? ? ? ? ? ? ? ?ws.onopen = function() { > ? ? ? ? ? ? ? ? ? ? ? ?alert("open"); > ? ? ? ? ? ? ? ? ? ?ws.send("Hello, Server!"); > ? ? ? ? ? ? ? ?}; > ? ? ? ? ? ?} From yujiangw@REDACTED Fri Mar 11 15:24:50 2011 From: yujiangw@REDACTED (Edward Wang) Date: Fri, 11 Mar 2011 22:24:50 +0800 Subject: Regarding clearing gen_server state In-Reply-To: References: Message-ID: FYI. This is a nice read and clarify the question for me: "On Erlang, State and Crashes" https://gist.github.com/85007bfc4f7b8c75ad8e On Sun, Feb 20, 2011 at 4:40 AM, Edward Wang wrote: > Hi, > > In Erlang, it is quite ubiquitous to use state of dynamically spawned > gen_server to represent entities in problem domain. These gen_servers can > trap exit signal and do housekeeping when gracefully terminating. If some of > them crash, a monitoring process can take care of housekeeping for them. > But, the problem is most likely the monitoring process doesn't have enough > data to do so. No shared state, remember? > > So, I end up with two copy of the same data, one in gen_server's state, one > in ets table. Not only I need to keep them in sync, but also it *is* shared > data to some extent. > > How can I do it differently? > > Regards, > Edward > From ericbmerritt@REDACTED Fri Mar 11 16:42:22 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Fri, 11 Mar 2011 09:42:22 -0600 Subject: [erlang-questions] Escript to run erlang applcation In-Reply-To: <1299822102670-3347510.post@n4.nabble.com> References: <1299822102670-3347510.post@n4.nabble.com> Message-ID: Escript doesn't have anything built in to do this for you. However, assuming it is a single app and its in your path you can just do application start. That is application:start(). Of course, that will only start up your application and will fail if your dependencies aren't started. The only option you really have then is to call application start on your entire dependency chain and your application in dependency order. Thats actually not as bad as it sounds. Of course after a while you are going to want to start releases via escript, but thats not really possible at the moment. Eric On Thu, Mar 10, 2011 at 11:41 PM, shk wrote: > Hello, > > I have erlang *.app and some *.erl files. I compiled it. Hoc can i write > escript script to run erlang *.app? > > Thank you. > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Escript-to-run-erlang-applcation-tp3347510p3347510.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kuleshovmail@REDACTED Fri Mar 11 18:49:13 2011 From: kuleshovmail@REDACTED (shk) Date: Fri, 11 Mar 2011 09:49:13 -0800 (PST) Subject: Parse TO: filed in mime Message-ID: <1299865753210-3348705.post@n4.nabble.com> Hello, I try to get TO: user@REDACTED field from mime mail message. I have code: parse_to(Data) -> List = string:tokens(Data, ":"), Sep1 = lists:map(fun(H) ->string:tokens(H, ":") end, List), io:format(Sep1), Sep2 = lists:filter(fun ([K | _]) -> K == "To" end, Sep1), ListAddress = lists:append(Sep2), [_ | Tail] = ListAddress, lists:map(fun(Address) -> string:tokens(Address, ",") end, Tail). If i have short message for example: https://gist.github.com/865910 I got in io:format(Sep1) https://gist.github.com/865905, it's ok all without : But if i have long message with attachment: - https://gist.github.com/865914 I got in io:format(Sep1) - https://gist.github.com/865906 everything remains the same as it was with : What's wrong? Why shot message normal parse and big message not parsed? When i try use regexp: List = binary_to_list(Binary), re:run(List, "^To: (.*)$", [multiline, {capture, all_but_first, list}]). It's ok, but if i get Binary from file. If i get from erlang list []. Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Parse-TO-filed-in-mime-tp3348705p3348705.html Sent from the Erlang Questions mailing list archive at Nabble.com. From yrashk@REDACTED Fri Mar 11 19:01:39 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Fri, 11 Mar 2011 10:01:39 -0800 Subject: [erlang-questions] Two beautiful programs - or web programming made easy In-Reply-To: References: <30893524.706.1297814455676.JavaMail.geo-discussion-forums@prie35> Message-ID: Just in case, early but transport-complete version of socket.io server for erlang is now available on GitHub: https://github.com/yrashk/socket.io-erlang We'll be adding more documentation and some nice features soon; but it seems to be quite usable as is :) On Wed, Feb 16, 2011 at 12:36 AM, Joe Armstrong wrote: > Sounds like a good idea - I didn't know about socket.io. I'll add it to my > "look-at-this-when-you-get-time" > list - thanks for the tip > > /Joe > > > On Wed, Feb 16, 2011 at 1:00 AM, Yurii Rashkovskii wrote: >> >> Wow, what a thread :) I just had a thought ? wouldn't it be more >> productive to do something like, for example, a port of socket.io >> (http://socket.io/) server to Erlang? (with misultin/mochiweb/yaws/.. >> supported) This way we can get WebSockets API pretty much regardless of the >> browser in question; it can also >> help luring some JS developers into Erlang since they'll be able to reuse >> their frontends for their Node.js services that don't scale. >> Basically, socket.io gives you that WebSocket API but fallbacks through a >> number of other methods until it finds one that works with a visitor's >> browser. >> I started some bits of it at?https://github.com/yrashk/socket.io-erlang >> but I am ultimately not able to keep up with all projects I am interested. >> So if anybody is interested, please let me know. >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From gregory.haskins@REDACTED Fri Mar 11 23:12:49 2011 From: gregory.haskins@REDACTED (Gregory Haskins) Date: Fri, 11 Mar 2011 17:12:49 -0500 Subject: [erlang-questions] Best practices for distributing an application over erlang? In-Reply-To: <38126.1297968825@snookles.snookles.com> References: <38126.1297968825@snookles.snookles.com> Message-ID: <4D7A9E61.6020204@gmail.com> On 2/17/11 1:53 PM, Scott Lystig Fritchie wrote: > Gregory Haskins wrote: > > gh> Long story short(er): I suspect I am just making all of this harder > gh> than it needs to be, and there must be a ton of you who are just > gh> doing similar operations all the time. > > Er, it seems to me that you are mixing together two problems: > > 1. managing application bugfixes & upgrades > 2. managing distributed application state > > Having a whiz-bang way to manage the BEAM files required to do #1 is > usually done by copying the new BEAM files to all the places that they > need to be, then using a. load a module at a time (usually via the > console, for small patches), b. OTP app upgrade (cool but tricky to > coordinate, depending on the app), or c. rolling node stop & restart. > > There are other ways to do it, as you were exploring, but they aren't > the sort of things that I'm aware of many (any?) people using in > production. (Aside from diskless/stateless embedded systems.) Or I'm > completely misunderstanding your question.... > Thanks for the detailed reply, Scott. I was afraid of your answer, but I started putting together a prototype to address this apparent shortcoming. I call it "EDist", short for erlang-distribution. If anyone is curious, I have the code up on github: https://github.com/ghaskins/edist Ill update the wiki with better notes on what it does today and what I want it to do in the future. Note that it is still very much in a prototype form, and I fully expect that the protocol ABI will change before I get to "1.0". Patches welcome, of course. Kind Regards, -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 267 bytes Desc: OpenPGP digital signature URL: From gregory.haskins@REDACTED Fri Mar 11 23:14:31 2011 From: gregory.haskins@REDACTED (Gregory Haskins) Date: Fri, 11 Mar 2011 17:14:31 -0500 Subject: What are the named-pipes in run_erl for? Message-ID: <4D7A9EC7.2080407@gmail.com> I cant seem to find any info on what these pipes are actually used for. Any details/pointers appreciated! Kind Regards, -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 267 bytes Desc: OpenPGP digital signature URL: From g@REDACTED Fri Mar 11 23:33:42 2011 From: g@REDACTED (Garrett Smith) Date: Fri, 11 Mar 2011 16:33:42 -0600 Subject: [erlang-questions] What are the named-pipes in run_erl for? In-Reply-To: <4D7A9EC7.2080407@gmail.com> References: <4D7A9EC7.2080407@gmail.com> Message-ID: It's how run_erl communicates with the Erlang process it starts. Try to_erl PIPEDIR/ to attach to the Erlang shell. You can issue commands by piping them through to_erl to the Erlang process as well. Pretty cool. echo "init:stop()." | to_erl PIPEDIR/ On Fri, Mar 11, 2011 at 4:14 PM, Gregory Haskins wrote: > I cant seem to find any info on what these pipes are actually used for. > ?Any details/pointers appreciated! > > Kind Regards, > -Greg > > From wolfmanjm@REDACTED Sat Mar 12 20:22:23 2011 From: wolfmanjm@REDACTED (Jim Morris) Date: Sat, 12 Mar 2011 11:22:23 -0800 (PST) Subject: Making an Ubuntu daemon out of a release while using heart In-Reply-To: References: Message-ID: I wrote a blog entry on how I do it... http://blog.wolfman.com/articles/2010/9/19/running-erlang-otp-applications-as-daemons-on-ubuntu-servers Maybe not the easiest, but it works for me. On Mar 9, 10:32?am, ori brost wrote: > I have a release of a server program in Erlang that I would like to turn > into a daemon. Currently I did this by writing a script (based on the > release management script generated by rebar) which runs the Erlang VM > synchronously, and writing an upstart .conf file which runs the script). > > My problem is that I want to use erl -heart. I have noticed that the > whenever the VM dies, the heart process restarts it, but then the VM starts > a new heart process and kills the old one. This means I have no permanent > PIDs to follow. I was thinking of implementing my solution by having the > script run a while-loop that will only exit if there is no heart process in > the system (for now, assume there is no other erlang program running on the > machine), and have upstart think that the script is the daemon. > > Does anyone have recommendation for a better solution? Or some general > recommendation on how to make a daemon out of an Erlang server? Preferably > using upstart? > > Best Regards, > > Ori Bar, Software Engineer > Nivertech Ltd > > email: ? ?ori....@REDACTED From bob@REDACTED Sun Mar 13 11:03:57 2011 From: bob@REDACTED (Bob Cowdery) Date: Sun, 13 Mar 2011 10:03:57 +0000 Subject: [erlang-questions] Binary manipulation In-Reply-To: <4D79E4F2.2010906@bobcowdery.plus.com> References: <4D79E4F2.2010906@bobcowdery.plus.com> Message-ID: <4D7C968D.4090206@bobcowdery.plus.com> Having received only one private reply which would not really scale up and having looked quite closely at all the API's available I can't see any way to manipulate binary data with any efficiency. I am therefore thinking of doing a NIF library. However I'm a bit concerned about what the following statement means: "Avoid doing lengthy work in NIF calls as that may degrade the responsiveness of the VM. NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns." I presume this just means the calling process cannot do any other work until the NIF returns and not that all processes can suffer degradation. Thanks Bob On 11/03/2011 09:01, Bob Cowdery wrote: > Hi > > I would appreciate any advice/ideas on how to efficiently handle a large > binary. > > The binary in question has a couple of protocol wrappers around it which > are no problem as the bit syntax easily copes with that. What I'm not > sure of is how to handle the data. > > <> > > What I have to do is extract I and Q which are 24 bit le values and end > up with another binary which is: > > <> > > Where I and Q are now floating point numbers. I will need M in the > future, but not right now. It's a little more complicated than that > because several of the raw data binaries will go into the processed > binary which will be a multiple of 64 up to 1024. > > I will also need to do the opposite, i.e. take the processed binary and > transform it back to the raw data binary. > > Thanks > Bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From lukas.larsson@REDACTED Sun Mar 13 11:22:49 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Sun, 13 Mar 2011 10:22:49 +0000 (GMT) Subject: [erlang-questions] Binary manipulation In-Reply-To: <382396153.54581300011536498.JavaMail.root@zimbra> Message-ID: <361084166.54601300011769241.JavaMail.root@zimbra> If you do write a nif call which could potentially take a long time you should use an async thread to do the work in it rather than doing it in the erlang vm thread. If not you could get some strange behaviour out of the vm depending on how unlucky you are with processes scheduling and unrelated work which other processes do. See http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:55980:201101:jconogbffcaogeijbdkl for details. Lukas ----- Original Message ----- From: "Bob Cowdery" To: erlang-questions@REDACTED Sent: Sunday, March 13, 2011 11:03:57 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: [erlang-questions] Binary manipulation Having received only one private reply which would not really scale up and having looked quite closely at all the API's available I can't see any way to manipulate binary data with any efficiency. I am therefore thinking of doing a NIF library. However I'm a bit concerned about what the following statement means: "Avoid doing lengthy work in NIF calls as that may degrade the responsiveness of the VM. NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns." I presume this just means the calling process cannot do any other work until the NIF returns and not that all processes can suffer degradation. Thanks Bob On 11/03/2011 09:01, Bob Cowdery wrote: > Hi > > I would appreciate any advice/ideas on how to efficiently handle a large > binary. > > The binary in question has a couple of protocol wrappers around it which > are no problem as the bit syntax easily copes with that. What I'm not > sure of is how to handle the data. > > <> > > What I have to do is extract I and Q which are 24 bit le values and end > up with another binary which is: > > <> > > Where I and Q are now floating point numbers. I will need M in the > future, but not right now. It's a little more complicated than that > because several of the raw data binaries will go into the processed > binary which will be a multiple of 64 up to 1024. > > I will also need to do the opposite, i.e. take the processed binary and > transform it back to the raw data binary. > > Thanks > Bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From michael.santos@REDACTED Sun Mar 13 12:09:20 2011 From: michael.santos@REDACTED (Michael Santos) Date: Sun, 13 Mar 2011 07:09:20 -0400 Subject: [erlang-questions] Binary manipulation In-Reply-To: <20110313110747.GC3664@ecn.lan> References: <4D79E4F2.2010906@bobcowdery.plus.com> <20110313110747.GC3664@ecn.lan> Message-ID: <20110313110920.GA5082@ecn.lan> On Fri, Mar 11, 2011 at 09:01:38AM +0000, Bob Cowdery wrote: > Hi > > I would appreciate any advice/ideas on how to efficiently handle a large > binary. > > The binary in question has a couple of protocol wrappers around it which > are no problem as the bit syntax easily copes with that. What I'm not > sure of is how to handle the data. > > <> > > What I have to do is extract I and Q which are 24 bit le values and end > up with another binary which is: > > <> > > Where I and Q are now floating point numbers. I will need M in the > future, but not right now. It's a little more complicated than that > because several of the raw data binaries will go into the processed > binary which will be a multiple of 64 up to 1024. > > I will also need to do the opposite, i.e. take the processed binary and > transform it back to the raw data binary. Maybe something like this would work: % Using a list m(Bin) -> m(Bin, []). % Assume big endian m(<<>>, Acc) -> list_to_binary(lists:reverse(Acc)); m(<>, Acc) -> m(Rest, [<>|Acc]). % Or constructing a binary each time m1(Bin) -> m1(Bin, <<>>). m1(<<>>, Acc) -> Acc; m1(<>, Acc) -> m1(Rest, <>). From klaus.trainer@REDACTED Sun Mar 13 18:09:47 2011 From: klaus.trainer@REDACTED (Klaus Trainer) Date: Sun, 13 Mar 2011 18:09:47 +0100 Subject: [erlang-questions] Binary manipulation In-Reply-To: <4D7C968D.4090206@bobcowdery.plus.com> References: <4D79E4F2.2010906@bobcowdery.plus.com> <4D7C968D.4090206@bobcowdery.plus.com> Message-ID: <1300036187.20976.31.camel@devil> > "Avoid doing lengthy work in NIF calls as that may degrade the > responsiveness of the VM. NIFs are called directly by the same > scheduler > thread that executed the calling Erlang code. The calling scheduler > will > thus be blocked from doing any other work until the NIF returns." > > I presume this just means the calling process cannot do any other work > until the NIF returns and not that all processes can suffer > degradation. No. If you have only one scheduler, it obviously means that no other process will be executed, because the only scheduler is blocked by your NIF. If you don't set a different configuration, the number of schedulers corresponds to the number of available processors. From pablo.platt@REDACTED Sun Mar 13 20:13:29 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 13 Mar 2011 12:13:29 -0700 (PDT) Subject: better error message when using term_to_binary with [safe] ? Message-ID: <123560.77083.qm@web112610.mail.gq1.yahoo.com> Hi, When using binary_to_term with the [safe] option, with a non existing atom it just throws an error but doesn't tell you what's wrong. Is it possible that the error message will tell me what atom was missing? Example: 1> binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>, [safe]). ** exception error: bad argument in function binary_to_term/2 called as binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>,[safe]) What is the best way to tell erlang what are the 'safe' atoms in my program? I'm currently defining and calling a trusted_atoms/0 function that just returns a list of all atoms I need. Is there a 'cleaner' way to do that? init() -> A = trusted_atoms(). trusted_atoms() -> [atom1, atom2 atom3]. From qwertymaniac@REDACTED Sun Mar 13 21:21:31 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Mon, 14 Mar 2011 01:51:31 +0530 Subject: [erlang-questions] better error message when using term_to_binary with [safe] ? In-Reply-To: <123560.77083.qm@web112610.mail.gq1.yahoo.com> References: <123560.77083.qm@web112610.mail.gq1.yahoo.com> Message-ID: Hello Pablo, On Mon, Mar 14, 2011 at 12:43 AM, Pablo Platt wrote: > Hi, > > When using binary_to_term with the [safe] option, with a non existing atom it > just throws an error > but doesn't tell you what's wrong. > Is it possible that the error message will tell me what atom was missing? > Example: > > 1> binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>, [safe]). > ** exception error: bad argument > ? ? in function ?binary_to_term/2 > ? ? ? ?called as binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>,[safe]) As the docs for this function note (in the manual), binary_to_term/2 will fail with a badarg if safe is specified and unsafe data is decoded. The trace also contains the exact binary data it was called with (which you can use to know what it really was, while debugging). It shall always show the exact binary content, even if used with a name: 1> A = <<131,100,0,6,109,121,97,116,111,109>>. 2> binary_to_term(A, [safe]). ** exception error: bad argument in function binary_to_term/2 called as binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>,[safe]) 3> binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>). myatom > What is the best way to tell erlang what are the 'safe' atoms in my program? By simply making the erlang runtime know it is not a 'new' atom. > I'm currently defining and calling a trusted_atoms/0 function that just returns > a list of all atoms I need. > Is there a 'cleaner' way to do that? > > init() -> > ? ?A = trusted_atoms(). > > trusted_atoms() -> > ? ?[atom1, atom2 atom3]. This is good enough I think. But I'll let others answer as to what'd be best. -- Harsh J www.harshj.com From pablo.platt@REDACTED Sun Mar 13 21:35:56 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 13 Mar 2011 13:35:56 -0700 (PDT) Subject: [erlang-questions] better error message when using term_to_binary with [safe] ? In-Reply-To: References: <123560.77083.qm@web112610.mail.gq1.yahoo.com> Message-ID: <299480.35323.qm@web112612.mail.gq1.yahoo.com> When you have a large binary with a complex erlang term that contains many atoms it can be hard to guess which atom cause the function to fail. I can write a function that traverses the binary and check every single atom but I would expect binary_to_term to just tell why it fails. Unless it is very hard to implement for some reason. ________________________________ From: Harsh J To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Sun, March 13, 2011 10:21:31 PM Subject: Re: [erlang-questions] better error message when using term_to_binary with [safe] ? Hello Pablo, On Mon, Mar 14, 2011 at 12:43 AM, Pablo Platt wrote: > Hi, > > When using binary_to_term with the [safe] option, with a non existing atom it > just throws an error > but doesn't tell you what's wrong. > Is it possible that the error message will tell me what atom was missing? > Example: > > 1> binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>, [safe]). > ** exception error: bad argument > in function binary_to_term/2 > called as binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>,[safe]) As the docs for this function note (in the manual), binary_to_term/2 will fail with a badarg if safe is specified and unsafe data is decoded. The trace also contains the exact binary data it was called with (which you can use to know what it really was, while debugging). It shall always show the exact binary content, even if used with a name: 1> A = <<131,100,0,6,109,121,97,116,111,109>>. 2> binary_to_term(A, [safe]). ** exception error: bad argument in function binary_to_term/2 called as binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>,[safe]) 3> binary_to_term(<<131,100,0,6,109,121,97,116,111,109>>). myatom > What is the best way to tell erlang what are the 'safe' atoms in my program? By simply making the erlang runtime know it is not a 'new' atom. > I'm currently defining and calling a trusted_atoms/0 function that just returns > a list of all atoms I need. > Is there a 'cleaner' way to do that? > > init() -> > A = trusted_atoms(). > > trusted_atoms() -> > [atom1, atom2 atom3]. This is good enough I think. But I'll let others answer as to what'd be best. -- Harsh J www.harshj.com From seancribbs@REDACTED Sun Mar 13 23:25:14 2011 From: seancribbs@REDACTED (Sean Cribbs) Date: Sun, 13 Mar 2011 18:25:14 -0400 Subject: [ANN] Neotoma 1.5 Message-ID: It's been a year coming, so I'm proud to announce a new version of Neotoma, the PEG/Packrat toolkit and parser-generator. The primary feature of version 1.5 is that it uses significantly less memory and also supports UTF8 (via binaries), thanks to some awesome work by Graeme Defty, who has also converted Reia to use a PEG. You can get the latest source on Github: https://github.com/seancribbs/neotoma Neotoma is rebar-friendly as well, so adding it to a rebar project is a simple deps declaration like so: {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, "1.5"}}} Hope to see some of you at Erlang Factory SF next week. Cheers, Sean Cribbs From sam@REDACTED Mon Mar 14 07:00:09 2011 From: sam@REDACTED (Sam Bobroff) Date: Mon, 14 Mar 2011 17:00:09 +1100 Subject: [erlang-questions] Making an Ubuntu daemon out of a release while using heart In-Reply-To: References: Message-ID: <4D7DAEE9.3020205@m5net.com> On 10/03/11 05:32, ori brost wrote: > I have a release of a server program in Erlang that I would like to turn > into a daemon. Currently I did this by writing a script (based on the > release management script generated by rebar) which runs the Erlang VM > synchronously, and writing an upstart .conf file which runs the script). > > My problem is that I want to use erl -heart. I have noticed that the > whenever the VM dies, the heart process restarts it, but then the VM starts > a new heart process and kills the old one. This means I have no permanent > PIDs to follow. I was thinking of implementing my solution by having the > script run a while-loop that will only exit if there is no heart process in > the system (for now, assume there is no other erlang program running on the > machine), and have upstart think that the script is the daemon. > > Does anyone have recommendation for a better solution? Or some general > recommendation on how to make a daemon out of an Erlang server? Preferably > using upstart? > > Best Regards, > > Ori Bar, Software Engineer > Nivertech Ltd > > email: ori.bar@REDACTED > Hi Ori, We faced similar problems when deploying an Erlang application to RedHat machines and ended up writing our own replacement for "run_erl" and "heart" (as a single program) that we call "erld". If you're interested I could probably send you the source (we're intending to release it as open source eventually anyway). It's intended for running an Erlang VM as a "well behaved" Unix daemon. The problems we hit (and which it solves), are, from memory: * PID confusion: The only PID that a unix init script can get hold of when starting Erlang is the PID of the beam.smp process, and when that is restarted by heart, it changes and the init script can't know what process to stop or kill. run_erl doesn't seem to support a "pidfile" option. * Zombie Erlang: If heart dies (or is killed), Erlang will beam.smp will restart it. If beam.smp dies, heart will restart it. This makes it rather hard to kill a system that is crashing badly (in fact if you don't know how to stop-but-not-kill a unix process you may have to reboot the server!). * Bad detach behavior: run_erl detaches from the console before it's actually started anything so an init script can't possibly report "OK" or "FAIL"... it just has to say "OK" and hope that the program has started. Admins have to check the logs every time to see if it's worked. (This is REALLY annoying to our system admins.) Properly behaved daemons won't detach until after they've started up *successfully*. Any fatal errors should be reported to the console. * Which log is current? The log rotation system that's built in has a bizarre rotation strategy that we don't like (the .1 .2 .4 .5 thing that you've probably seen). * Because run_erl will happily created any number of sockets in /tmp, it's easy to accidentally start multiple (probably non-functional) copies of your application. It also does a few other things: * Reopens log files on SIGHUP. * Captures any stdout from the Erlang VM. I suspect even fairly big Erlang Unix apps face these problems, but I haven't yet seen any other solutions (e.g. ejabberd, yaws). Does anyone know? Peace, Sam. From jocke@REDACTED Mon Mar 14 07:54:09 2011 From: jocke@REDACTED (Joakim G.) Date: Mon, 14 Mar 2011 07:54:09 +0100 Subject: no default value support in xmerl? Message-ID: <4D7DBB91.7040309@tail-f.com> The XML schema support in xmerl considers all default elements as mandatory. If this is true it is a bug of a grander scale. ================================================================ Take for example test.xsd: ================================================================ and test.xml: 42 ================================================================ then you will see: 28> xmerl_scan:file("/tmp/test.xml", [{validation, schema}, {schemaLocation, [{"foo", "/tmp/test.xsd"}]}]). 3940- fatal: {failed_schema_validation, [{[],xmerl_xsd, {missing_mandatory_elements, {element,{{bar,[anonymous,foo],[]},{1,1}}}}}]} ** exception exit: {fatal, {{failed_schema_validation, [{[],xmerl_xsd, {missing_mandatory_elements, {element, {{bar,[anonymous,foo],[]},{1,1}}}}}]}, {file,"/tmp/test.xml"}, {line,4}, {col,1}}} in function xmerl_scan:fatal/2 in call from xmerl_scan:scan_document/2 in call from xmerl_scan:file/2 29> ================================================================ Default values are not mandatory AFAIK. I might be doing something wrong. It has happened before. Cheers /Jocke From gordon@REDACTED Mon Mar 14 09:31:45 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Mon, 14 Mar 2011 08:31:45 +0000 Subject: VM ends with 'killed' Message-ID: Folks I have been trying to resolve some run time issues. We are running a big data load and the VM was crashing due to memory running out. The first issue was a runaway message queue - so we put in a limiter that checked that processes message queue and waited til it cleared if there was overload. The problem moved to another process whose heap would grow and grow. Adding a check on the heap size and forced garbage collect appeared to be the answer. But now when I run the load, after adding most of the data the VM suddenly ends with the stark message: Killed [End] There is no crash dump. What do I do now? Where do I start debugging this? Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From bob@REDACTED Mon Mar 14 10:44:45 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 14 Mar 2011 09:44:45 +0000 Subject: [erlang-questions] Binary manipulation In-Reply-To: <361084166.54601300011769241.JavaMail.root@zimbra> References: <361084166.54601300011769241.JavaMail.root@zimbra> Message-ID: <4D7DE38D.4040005@bobcowdery.plus.com> Thanks. Sounds like I should move all the processing down into a threaded C NIF. So I would pass back, in the case of decoding an empty binary or a binary which is the multiple of 64 required. This can easily be done on a separate thread as there are several calls to build one response. Opposite on the encoding bit, pass back the largest list of binaries that can be encoded from the multiple of 64 passed in. However, this amounts to a polling loop from the erlang process waiting for the result. That doesn't sound so great. Another alternative is to do all that in a linked-in driver as it is a big chuck of C code that is processing these large binaries anyway. Downside is that it breaks good program structure. Bob On 13/03/2011 10:22, Lukas Larsson wrote: > If you do write a nif call which could potentially take a long time you should use an async thread to do the work in it rather than doing it in the erlang vm thread. If not you could get some strange behaviour out of the vm depending on how unlucky you are with processes scheduling and unrelated work which other processes do. See http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:55980:201101:jconogbffcaogeijbdkl for details. > > Lukas > > > ----- Original Message ----- > From: "Bob Cowdery" > To: erlang-questions@REDACTED > Sent: Sunday, March 13, 2011 11:03:57 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna > Subject: Re: [erlang-questions] Binary manipulation > > Having received only one private reply which would not really scale up > and having looked quite closely at all the API's available I can't see > any way to manipulate binary data with any efficiency. I am therefore > thinking of doing a NIF library. However I'm a bit concerned about what > the following statement means: > > "Avoid doing lengthy work in NIF calls as that may degrade the > responsiveness of the VM. NIFs are called directly by the same scheduler > thread that executed the calling Erlang code. The calling scheduler will > thus be blocked from doing any other work until the NIF returns." > > I presume this just means the calling process cannot do any other work > until the NIF returns and not that all processes can suffer degradation. > > Thanks > Bob > > On 11/03/2011 09:01, Bob Cowdery wrote: >> Hi >> >> I would appreciate any advice/ideas on how to efficiently handle a large >> binary. >> >> The binary in question has a couple of protocol wrappers around it which >> are no problem as the bit syntax easily copes with that. What I'm not >> sure of is how to handle the data. >> >> <> >> >> What I have to do is extract I and Q which are 24 bit le values and end >> up with another binary which is: >> >> <> >> >> Where I and Q are now floating point numbers. I will need M in the >> future, but not right now. It's a little more complicated than that >> because several of the raw data binaries will go into the processed >> binary which will be a multiple of 64 up to 1024. >> >> I will also need to do the opposite, i.e. take the processed binary and >> transform it back to the raw data binary. >> >> Thanks >> Bob >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ttom.kelly@REDACTED Mon Mar 14 11:05:40 2011 From: ttom.kelly@REDACTED (tom kelly) Date: Mon, 14 Mar 2011 10:05:40 +0000 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: Hi Gordon, We had a crash matching this description on our system a while ago. We got some good hints as to what was happening from the system logs, especially: /var/log/messages /var/log/syslog In our case we were getting segmentation faults. We had been sending more data in a TCP packet than would fit into it with the "packet" option we had provided. We dismissed this as a cause for ages as we were catching the exit message when we did a binary_to _term on the data. Once we figured out what was happening we consulted the release notes and found out that the Erlang VM had already been fixed so it's probably not your problem. Hope this helps. //Tom. On Mon, Mar 14, 2011 at 8:31 AM, Gordon Guthrie wrote: > Folks > > I have been trying to resolve some run time issues. We are running a > big data load and the VM was crashing due to memory running out. > > The first issue was a runaway message queue - so we put in a limiter > that checked that processes message queue and waited til it cleared if > there was overload. > > The problem moved to another process whose heap would grow and grow. > Adding a check on the heap size and forced garbage collect appeared to > be the answer. > > But now when I run the load, after adding most of the data the VM > suddenly ends with the stark message: > Killed > [End] > > There is no crash dump. What do I do now? Where do I start debugging this? > > Gordon > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From yujiangw@REDACTED Mon Mar 14 11:15:34 2011 From: yujiangw@REDACTED (Edward Wang) Date: Mon, 14 Mar 2011 18:15:34 +0800 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: http://erlang.2086793.n4.nabble.com/Patch-for-18-exabyte-memory-allocation-failure-td3342133.html This may be related. Edward. On Mar 14, 2011 6:08 PM, "tom kelly" wrote: > Hi Gordon, > > We had a crash matching this description on our system a while ago. > We got some good hints as to what was happening from the system logs, > especially: > /var/log/messages > /var/log/syslog > > In our case we were getting segmentation faults. We had been sending more > data in a TCP packet than would fit into it with the "packet" option we had > provided. We dismissed this as a cause for ages as we were catching the exit > message when we did a binary_to _term on the data. > Once we figured out what was happening we consulted the release notes and > found out that the Erlang VM had already been fixed so it's probably not > your problem. > > Hope this helps. > //Tom. > > > On Mon, Mar 14, 2011 at 8:31 AM, Gordon Guthrie wrote: > >> Folks >> >> I have been trying to resolve some run time issues. We are running a >> big data load and the VM was crashing due to memory running out. >> >> The first issue was a runaway message queue - so we put in a limiter >> that checked that processes message queue and waited til it cleared if >> there was overload. >> >> The problem moved to another process whose heap would grow and grow. >> Adding a check on the heap size and forced garbage collect appeared to >> be the answer. >> >> But now when I run the load, after adding most of the data the VM >> suddenly ends with the stark message: >> Killed >> [End] >> >> There is no crash dump. What do I do now? Where do I start debugging this? >> >> Gordon >> >> -- >> Gordon Guthrie >> CEO hypernumbers >> >> http://hypernumbers.com >> t: hypernumbers >> +44 7776 251669 >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From rtrlists@REDACTED Mon Mar 14 11:19:22 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 14 Mar 2011 10:19:22 +0000 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: Hi Gordon, On Mon, Mar 14, 2011 at 8:31 AM, Gordon Guthrie wrote: > Folks > > I have been trying to resolve some run time issues. We are running a > big data load and the VM was crashing due to memory running out. > > The first issue was a runaway message queue - so we put in a limiter > that checked that processes message queue and waited til it cleared if > there was overload. > > The problem moved to another process whose heap would grow and grow. > Adding a check on the heap size and forced garbage collect appeared to > be the answer. > > But now when I run the load, after adding most of the data the VM > suddenly ends with the stark message: > Killed > [End] > > There is no crash dump. What do I do now? Where do I start debugging this? > > Gordon > > This'll be on Linux, yes? I've seen this happen when Linux decides that any one of the processes just needs to die, in order for the system as a whole to continue. If I remember correctly it has to do with running out of physical memory, and Linux as a last resort picks victim to kill. And it kills it outright. There may be heuristics as to which process it chooses. Robby From taylor@REDACTED Mon Mar 14 11:34:15 2011 From: taylor@REDACTED (Taylor Venable) Date: Mon, 14 Mar 2011 06:34:15 -0400 Subject: [erlang-questions] no default value support in xmerl? In-Reply-To: <4D7DBB91.7040309@tail-f.com> References: <4D7DBB91.7040309@tail-f.com> Message-ID: On Mon, Mar 14, 2011 at 02:54, Joakim G. wrote: > The XML schema support in xmerl considers all default elements > as mandatory. I think this is correct behavior; according to http://www.w3.org/TR/xmlschema-0/ "The schema processor treats defaulted elements slightly differently. When an element is declared with a default value, the value of the element is whatever value appears as the element's content in the instance document; if the element appears without any content, the schema processor provides the element with a value equal to that of the default attribute. However, if the element does not appear in the instance document, the schema processor does not provide the element at all. In summary, the differences between element and attribute defaults can be stated as: Default attribute values apply when attributes are missing, and default element values apply when elements are empty." -- Taylor C. Venable http://metasyntax.net/ From icfp.publicity@REDACTED Mon Mar 14 11:51:27 2011 From: icfp.publicity@REDACTED (Wouter Swierstra) Date: Mon, 14 Mar 2011 11:51:27 +0100 Subject: ICFP 2011 Deadline Extension Message-ID: ===================================================================== ICFP 2011: International Conference on Functional Programming http://www.icfpconference.org/icfp2011 ===================================================================== On behalf of the Program Committee of ICFP 2011, I would like to announce a deadline extension for authors affected by the recent earthquake in Japan. The Program Committee will accept titles and abstracts until Monday 4 April at 23:59, and full submissions until Thursday 7 April at 23:59 from those authors affected by the earthquake. Our thoughts go out to the victims of this tragedy. Wouter Swierstra ICFP Publicity Chair From watson.timothy@REDACTED Mon Mar 14 12:08:12 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 14 Mar 2011 11:08:12 +0000 Subject: [erlang-questions] RE: R14 vs R13 binary handling in driver_outputv In-Reply-To: References: <0711D4C9-8E9B-4DAD-AB96-1A26F608A965@gmail.com> Message-ID: > > Fixed. Now can anyone suggest a better way of consuming the binary in > driver_outputv? > On further reading of the erl_driver source code, I can see I'm doing this half baked. The obvious (and simple) solution is to use the iov_len on each member to get the length, instead of passing it explicitly. Hopefully by posting the answer, anyone looking in future will be able to pick up on this. Based on the post at http://www.erlang.org/pipermail/erlang-questions/2006-August/021897.html, a little fiddling around with the inputs and it's clear that you can just pass the binaries (i.e. [Xml, Xsl] and then some parameters if you wish) to port_command and then use the vsize and iov_len (or binv[x]->orig_size) to figure out the bounds and access the data. From jesper.louis.andersen@REDACTED Mon Mar 14 13:08:38 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 14 Mar 2011 13:08:38 +0100 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: On Mon, Mar 14, 2011 at 11:19, Robert Raschke wrote: > This'll be on Linux, yes? I've seen this happen when Linux decides that any > one of the processes just needs to die, in order for the system as a whole > to continue. In out of memory situations, the OOM-killer gets to work, grabs its scythe and reaps a process on Linux. But more often today, I see user limits set (ulimit -a in a shell) which limits how much memory you are going to get. This is especially true on virtual machines with contrained memory. It is usually just a "killed" message you end up with in that case. There probably still is a memory leak somewhere in the application. -- J. From jocke@REDACTED Mon Mar 14 14:16:27 2011 From: jocke@REDACTED (Joakim G.) Date: Mon, 14 Mar 2011 14:16:27 +0100 Subject: [erlang-questions] no default value support in xmerl? In-Reply-To: References: <4D7DBB91.7040309@tail-f.com> Message-ID: <4D7E152B.4020604@tail-f.com> I be damned! Haha. It's XML Schema that is broken then. Smile. Thanks /Jocke On 2011-03-14 11:34, Taylor Venable wrote: > On Mon, Mar 14, 2011 at 02:54, Joakim G. wrote: >> The XML schema support in xmerl considers all default elements >> as mandatory. > > I think this is correct behavior; according to http://www.w3.org/TR/xmlschema-0/ > > "The schema processor treats defaulted elements slightly differently. > When an element is declared with a default value, the value of the > element is whatever value appears as the element's content in the > instance document; if the element appears without any content, the > schema processor provides the element with a value equal to that of > the default attribute. However, if the element does not appear in the > instance document, the schema processor does not provide the element > at all. In summary, the differences between element and attribute > defaults can be stated as: Default attribute values apply when > attributes are missing, and default element values apply when elements > are empty." > From mevans@REDACTED Mon Mar 14 14:28:35 2011 From: mevans@REDACTED (Evans, Matthew) Date: Mon, 14 Mar 2011 09:28:35 -0400 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: Are you running with heart? If so you may want to do your test with heart disabled. I've seen that when heart is enabled it often stops the crash dump from getting generated. >From experience though, my guess would be a memory leak in one or more processes. Matt -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Gordon Guthrie Sent: Monday, March 14, 2011 4:32 AM To: erlang-questions Questions Subject: [erlang-questions] VM ends with 'killed' Folks I have been trying to resolve some run time issues. We are running a big data load and the VM was crashing due to memory running out. The first issue was a runaway message queue - so we put in a limiter that checked that processes message queue and waited til it cleared if there was overload. The problem moved to another process whose heap would grow and grow. Adding a check on the heap size and forced garbage collect appeared to be the answer. But now when I run the load, after adding most of the data the VM suddenly ends with the stark message: Killed [End] There is no crash dump. What do I do now? Where do I start debugging this? Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From ericbmerritt@REDACTED Mon Mar 14 15:23:06 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 14 Mar 2011 10:23:06 -0400 Subject: [erlang-questions] Making an Ubuntu daemon out of a release while using heart In-Reply-To: <4D7DAEE9.3020205@m5net.com> References: <4D7DAEE9.3020205@m5net.com> Message-ID: Sam, Let me just encourage you to release this. I have run into this need on several occasions and have a few adhoc approaches to it. Having a decent well tested solution out in the community would be a very good thing. Eric On Mon, Mar 14, 2011 at 2:00 AM, Sam Bobroff wrote: > On 10/03/11 05:32, ori brost wrote: >> I have a release of a server program in Erlang that I would like to turn >> into a daemon. Currently I did this by writing a script (based on the >> release management script generated by rebar) which runs the Erlang VM >> synchronously, and writing an upstart .conf file which runs the script). >> >> My problem is that I want to use erl -heart. I have noticed that the >> whenever the VM dies, the heart process restarts it, but then the VM starts >> a new heart process and kills the old one. This means I have no permanent >> PIDs to follow. I was thinking of implementing my solution by having the >> script run a while-loop that will only exit if there is no heart process in >> the system (for now, assume there is no other erlang program running on the >> machine), and have upstart think that the script is the daemon. >> >> Does anyone have recommendation for a better solution? Or some general >> recommendation on how to make a daemon out of an Erlang server? Preferably >> using upstart? >> >> Best Regards, >> >> Ori Bar, Software Engineer >> Nivertech Ltd >> >> email: ? ?ori.bar@REDACTED >> > Hi Ori, > > ? ?We faced similar problems when deploying an Erlang application to > RedHat machines and ended up writing our own replacement for "run_erl" > and "heart" (as a single program) that we call "erld". If you're > interested I could probably send you the source (we're intending to > release it as open source eventually anyway). It's intended for running > an Erlang VM as a "well behaved" Unix daemon. > > ? ?The problems we hit (and which it solves), are, from memory: > > * PID confusion: The only PID that a unix init script can get hold of > when starting Erlang is the PID of the beam.smp process, and when that > is restarted by heart, it changes and the init script can't know what > process to stop or kill. run_erl doesn't seem to support a "pidfile" option. > > * Zombie Erlang: If heart dies (or is killed), Erlang will beam.smp will > restart it. If beam.smp dies, heart will restart it. This makes it > rather hard to kill a system that is crashing badly (in fact if you > don't know how to stop-but-not-kill a unix process you may have to > reboot the server!). > > * Bad detach behavior: run_erl detaches from the console before it's > actually started anything so an init script can't possibly report "OK" > or "FAIL"... it just has to say "OK" and hope that the program has > started. Admins have to check the logs every time to see if it's worked. > (This is REALLY annoying to our system admins.) Properly behaved daemons > won't detach until after they've started up *successfully*. Any fatal > errors should be reported to the console. > > * Which log is current? The log rotation system that's built in has a > bizarre rotation strategy that we don't like (the .1 .2 .4 .5 thing that > you've probably seen). > > * Because run_erl will happily created any number of sockets in /tmp, > it's easy to accidentally start multiple (probably non-functional) > copies of your application. > > It also does a few other things: > * Reopens log files on SIGHUP. > * Captures any stdout from the Erlang VM. > > I suspect even fairly big Erlang Unix apps face these problems, but I > haven't yet seen any other solutions (e.g. ejabberd, yaws). Does anyone > know? > > Peace, > Sam. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From allen.kim@REDACTED Mon Mar 14 17:21:34 2011 From: allen.kim@REDACTED (Allen Kim) Date: Mon, 14 Mar 2011 11:21:34 -0500 Subject: [erlang-questions] Making an Ubuntu daemon out of a release while using heart In-Reply-To: <4D7DAEE9.3020205@m5net.com> Message-ID: +1 for early release. On 11-03-14 2:00 AM, "Sam Bobroff" wrote: >On 10/03/11 05:32, ori brost wrote: >> I have a release of a server program in Erlang that I would like to turn >> into a daemon. Currently I did this by writing a script (based on the >> release management script generated by rebar) which runs the Erlang VM >> synchronously, and writing an upstart .conf file which runs the script). >> >> My problem is that I want to use erl -heart. I have noticed that the >> whenever the VM dies, the heart process restarts it, but then the VM >>starts >> a new heart process and kills the old one. This means I have no >>permanent >> PIDs to follow. I was thinking of implementing my solution by having the >> script run a while-loop that will only exit if there is no heart >>process in >> the system (for now, assume there is no other erlang program running on >>the >> machine), and have upstart think that the script is the daemon. >> >> Does anyone have recommendation for a better solution? Or some general >> recommendation on how to make a daemon out of an Erlang server? >>Preferably >> using upstart? >> >> Best Regards, >> >> Ori Bar, Software Engineer >> Nivertech Ltd >> >> email: ori.bar@REDACTED >> >Hi Ori, > > We faced similar problems when deploying an Erlang application to >RedHat machines and ended up writing our own replacement for "run_erl" >and "heart" (as a single program) that we call "erld". If you're >interested I could probably send you the source (we're intending to >release it as open source eventually anyway). It's intended for running >an Erlang VM as a "well behaved" Unix daemon. > > The problems we hit (and which it solves), are, from memory: > >* PID confusion: The only PID that a unix init script can get hold of >when starting Erlang is the PID of the beam.smp process, and when that >is restarted by heart, it changes and the init script can't know what >process to stop or kill. run_erl doesn't seem to support a "pidfile" >option. > >* Zombie Erlang: If heart dies (or is killed), Erlang will beam.smp will >restart it. If beam.smp dies, heart will restart it. This makes it >rather hard to kill a system that is crashing badly (in fact if you >don't know how to stop-but-not-kill a unix process you may have to >reboot the server!). > >* Bad detach behavior: run_erl detaches from the console before it's >actually started anything so an init script can't possibly report "OK" >or "FAIL"... it just has to say "OK" and hope that the program has >started. Admins have to check the logs every time to see if it's worked. >(This is REALLY annoying to our system admins.) Properly behaved daemons >won't detach until after they've started up *successfully*. Any fatal >errors should be reported to the console. > >* Which log is current? The log rotation system that's built in has a >bizarre rotation strategy that we don't like (the .1 .2 .4 .5 thing that >you've probably seen). > >* Because run_erl will happily created any number of sockets in /tmp, >it's easy to accidentally start multiple (probably non-functional) >copies of your application. > >It also does a few other things: >* Reopens log files on SIGHUP. >* Captures any stdout from the Erlang VM. > >I suspect even fairly big Erlang Unix apps face these problems, but I >haven't yet seen any other solutions (e.g. ejabberd, yaws). Does anyone >know? > >Peace, >Sam. > > >________________________________________________________________ >erlang-questions (at) erlang.org mailing list. >See http://www.erlang.org/faq.html >To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From tony.arcieri@REDACTED Mon Mar 14 21:56:41 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Mon, 14 Mar 2011 14:56:41 -0600 Subject: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: References: Message-ID: Any plans for further performance improvements in the roadmap? :) On Sun, Mar 13, 2011 at 4:25 PM, Sean Cribbs wrote: > It's been a year coming, so I'm proud to announce a new version of > Neotoma, the PEG/Packrat toolkit and parser-generator. The primary > feature of version 1.5 is that it uses significantly less memory and > also supports UTF8 (via binaries), thanks to some awesome work by > Graeme Defty, who has also converted Reia to use a PEG. > > You can get the latest source on Github: > https://github.com/seancribbs/neotoma > > Neotoma is rebar-friendly as well, so adding it to a rebar project is > a simple deps declaration like so: > > {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, > "1.5"}}} > > Hope to see some of you at Erlang Factory SF next week. > > Cheers, > > Sean Cribbs > -- Tony Arcieri Medioh! Kudelski From comptekki@REDACTED Mon Mar 14 22:33:39 2011 From: comptekki@REDACTED (Wes James) Date: Mon, 14 Mar 2011 15:33:39 -0600 Subject: counter.erl example from "Concurrent Programming in ERLANG" - debugging Message-ID: I run the counter.erl in the shell - Pid=counter:start(). Then ran counter:value(Pid) - returns 0. I wanted to see how this works in the debugger so I run (after starting new shell or f() ): c(counter,[debug_info]). im(). ii(counter). iaa([init]). then Pid=counter:start(). It then runs through some of the code lines in the debugger. I then run counter:value(Pid), but then it just freezes after I press enter. Can the debugger not debug this? got out of the shell then back in and just ran up to counter:value(Pid) again and it returns 0. So the debugger is not liking something. thx, -wes From alain.odea@REDACTED Tue Mar 15 01:15:34 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 14 Mar 2011 21:45:34 -0230 Subject: [erlang-questions] counter.erl example from "Concurrent Programming in ERLANG" - debugging In-Reply-To: References: Message-ID: Hi Wes: On 2011-03-14, at 19:03, Wes James wrote: > I run the counter.erl in the shell - Pid=counter:start(). Then ran > counter:value(Pid) - returns 0. I wanted to see how this works in the > debugger so I run (after starting new shell or f() ): > > > c(counter,[debug_info]). > im(). > ii(counter). > iaa([init]). > > > then Pid=counter:start(). > > It then runs through some of the code lines in the debugger. I then > run counter:value(Pid), but then it just freezes after I press enter. > Can the debugger not debug this? > > got out of the shell then back in and just ran up to > counter:value(Pid) again and it returns 0. So the debugger is not > liking something. > > thx, > > -wes Out of curiosity, why are you using that particular book? Erlang and its tools may have changed enough to break it's examples. Much more current resources are available: Fr?d?ric Trottier-H?bert's Learn You Some Erlang for Great Good: http://learnyousomeerlang.com/ Joe Armstrong's Programming Erlang, Software for a Concurrent World: http://pragprog.com/titles/jaerlang/programming-erlang Martin Logan, Richard Carlsson and Eric Merritt's Erlang and OTP in Action: http://www.manning.com/logan/ Francesco Cesarini and Simon Thompson's Erlang Programming: http://oreilly.com/catalog/9780596518189 From comptekki@REDACTED Tue Mar 15 05:27:43 2011 From: comptekki@REDACTED (Wes James) Date: Mon, 14 Mar 2011 22:27:43 -0600 Subject: [erlang-questions] counter.erl example from "Concurrent Programming in ERLANG" - debugging In-Reply-To: References: Message-ID: On Mon, Mar 14, 2011 at 6:15 PM, Alain O'Dea wrote: > Hi Wes: Hi Alain: > On 2011-03-14, at 19:03, Wes James wrote: > > I run the counter.erl in the shell - Pid=counter:start(). Then ran > counter:value(Pid) - returns 0. I wanted to see how this works in the > debugger so I run (after starting new shell or f() ): > > Out of curiosity, why are you using that > particular book? Erlang and its tools may have changed enough to break it's > examples. Well about March of 2010 I found Erlang and I checked the University library here and they had this book and I checked it out. I ordered a copy from amazon and hope to get a few signatures next week at the SF Factory :) It's old, but the basics probably still hold. There might be a general issue with debugging spawns/receives, etc. Hopefully I'll have more input by tomorrow from some folks over the sea :) > Much more current resources are available: > Fr?d?ric Trottier-H?bert's Learn You Some Erlang for Great Good: > http://learnyousomeerlang.com/ I think I've been there a few times. > Joe Armstrong's Programming Erlang, Software for a Concurrent World: > http://pragprog.com/titles/jaerlang/programming-erlang Ordered it a couple of weeks ago and got it last week :) > Martin Logan, Richard Carlsson and Eric Merritt's Erlang and OTP in Action: > http://www.manning.com/logan/ > > Francesco Cesarini and Simon Thompson's Erlang Programming: > http://oreilly.com/catalog/9780596518189 I've seen these on the erlang.org side panel and might look in to them too. thx for the tips. -wes From gordon@REDACTED Tue Mar 15 09:08:56 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Tue, 15 Mar 2011 08:08:56 +0000 Subject: [erlang-questions] VM ends with 'killed' In-Reply-To: References: Message-ID: Folks You were all right. Basically the old system was badly behaved and would issue an extravagant demand for memory to be rebuffed by the O/S and then petulantly die leaving an erl_crash_dump with a 'cannot allocate XXXX of heap' message. With my improvements and garbage collecting the application was much better but the db simply got too big. Eventually Linux got tired of the VM and some other urchin of a process fighting over the crust of a half-chewed SIMM and delivered a resounding skelp... Same thing, different message... But some judicious erlang:term_to_binary(...) reduced the db size by 89% for the win.. We were running on a tichy cheapskate Linode anyhoo... Gordon On 14 March 2011 13:28, Evans, Matthew wrote: > Are you running with heart? > > If so you may want to do your test with heart disabled. I've seen that when heart is enabled it often stops the crash dump from getting generated. > > From experience though, my guess would be a memory leak in one or more processes. > > Matt > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Gordon Guthrie > Sent: Monday, March 14, 2011 4:32 AM > To: erlang-questions Questions > Subject: [erlang-questions] VM ends with 'killed' > > Folks > > I have been trying to resolve some run time issues. We are running a > big data load and the VM was crashing due to memory running out. > > The first issue was a runaway message queue - so we put in a limiter > that checked that processes message queue and waited til it cleared if > there was overload. > > The problem moved to another process whose heap would grow and grow. > Adding a check on the heap size and forced garbage collect appeared to > be the answer. > > But now when I run the load, after adding most of the data the VM > suddenly ends with the stark message: > Killed > [End] > > There is no crash dump. What do I do now? Where do I start debugging this? > > Gordon > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From dmitrii@REDACTED Tue Mar 15 10:33:42 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Tue, 15 Mar 2011 11:33:42 +0200 Subject: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: References: Message-ID: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> Awesome! Is reia grammar available? Could make a great place to start learning from > It's been a year coming, so I'm proud to announce a new version of > Neotoma, the PEG/Packrat toolkit and parser-generator. The primary > feature of version 1.5 is that it uses significantly less memory and > also supports UTF8 (via binaries), thanks to some awesome work by > Graeme Defty, who has also converted Reia to use a PEG. > > You can get the latest source on Github: https://github.com/seancribbs/neotoma > > Neotoma is rebar-friendly as well, so adding it to a rebar project is > a simple deps declaration like so: > > {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, "1.5"}}} > > Hope to see some of you at Erlang Factory SF next week. > > Cheers, > > Sean Cribbs From loris.fichera@REDACTED Tue Mar 15 11:54:31 2011 From: loris.fichera@REDACTED (Loris Fichera) Date: Tue, 15 Mar 2011 11:54:31 +0100 Subject: add a time interval to a timestamp Message-ID: <4D7F4567.5050801@gmail.com> Hello everybody, Say I want to write a function that takes as input a timestamp (in the form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by adding a certain amount of {hours, minutes, seconds} to the input. It wouldn't be difficult but, since I'm dealing with tuples, I wouldn't have any guarantee on the validity of the obtained timestamp, e.g. I could have obtained something like {{2011,02,30},{...}}. In such cases, the calendar:valid_date/1 and calendar:last_day_of_the_month/2 would be useful. I was just wondering if there already exists a function that does such a work. :) Thanks in advance. -- [ "let me take you down, 'cos i'm going to Strawberry Fields." ] [ Loris Fichera a.k.a. kid_a | HOME http://www.bakerstreet.it ] [ JABBER loris[dot]fichera[at]gmail[dot]com |GPG key ID 8B343E44 ] From qwertymaniac@REDACTED Tue Mar 15 12:22:13 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Tue, 15 Mar 2011 16:52:13 +0530 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: <4D7F4567.5050801@gmail.com> References: <4D7F4567.5050801@gmail.com> Message-ID: Hello, How about handling this with seconds since year 0? calendary:datetime_to_gregorian_seconds to get seconds, adding up this integer in seconds as you need and converting back to datetime with calendary:gregorian_seconds_to_datetime. On Tue, Mar 15, 2011 at 4:24 PM, Loris Fichera wrote: > Hello everybody, > > Say I want to write a function that takes as input a timestamp (in the > form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by > adding a certain amount of {hours, minutes, seconds} to the input. -- Harsh J http://harshj.com From loris.fichera@REDACTED Tue Mar 15 12:50:08 2011 From: loris.fichera@REDACTED (Loris Fichera) Date: Tue, 15 Mar 2011 12:50:08 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: <4D7F5270.6090507@gmail.com> That's a great suggestion, will make things easier. Thanks. :) On 03/15/2011 12:22 PM, Harsh J wrote: > Hello, > > How about handling this with seconds since year 0? > calendary:datetime_to_gregorian_seconds to get seconds, adding up this > integer in seconds as you need and converting back to datetime with > calendary:gregorian_seconds_to_datetime. > > On Tue, Mar 15, 2011 at 4:24 PM, Loris Fichera wrote: >> Hello everybody, >> >> Say I want to write a function that takes as input a timestamp (in the >> form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by >> adding a certain amount of {hours, minutes, seconds} to the input. > From alain.odea@REDACTED Tue Mar 15 13:12:13 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 15 Mar 2011 09:42:13 -0230 Subject: [erlang-questions] counter.erl example from "Concurrent Programming in ERLANG" - debugging In-Reply-To: References: Message-ID: <4EFD0DB1-03C0-4561-B741-436241D4F26A@gmail.com> Good stuff :) Just making sure you weren't led to believe that is the canonical book :) On 2011-03-15, at 1:57, Wes James wrote: > On Mon, Mar 14, 2011 at 6:15 PM, Alain O'Dea wrote: >> Hi Wes: > > Hi Alain: > >> On 2011-03-14, at 19:03, Wes James wrote: >> >> I run the counter.erl in the shell - Pid=counter:start(). Then ran >> counter:value(Pid) - returns 0. I wanted to see how this works in the >> debugger so I run (after starting new shell or f() ): > > >> >> Out of curiosity, why are you using that >> particular book? Erlang and its tools may have changed enough to break it's >> examples. > > Well about March of 2010 I found Erlang and I checked the University > library here and they had this book and I checked it out. I ordered a > copy from amazon and hope to get a few signatures next week at the SF > Factory :) It's old, but the basics probably still hold. There might > be a general issue with debugging spawns/receives, etc. Hopefully > I'll have more input by tomorrow from some folks over the sea :) > >> Much more current resources are available: >> Fr?d?ric Trottier-H?bert's Learn You Some Erlang for Great Good: >> http://learnyousomeerlang.com/ > > I think I've been there a few times. > >> Joe Armstrong's Programming Erlang, Software for a Concurrent World: >> http://pragprog.com/titles/jaerlang/programming-erlang > > Ordered it a couple of weeks ago and got it last week :) > >> Martin Logan, Richard Carlsson and Eric Merritt's Erlang and OTP in Action: >> http://www.manning.com/logan/ >> >> Francesco Cesarini and Simon Thompson's Erlang Programming: >> http://oreilly.com/catalog/9780596518189 > > I've seen these on the erlang.org side panel and might look in to them too. > > thx for the tips. > > -wes > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From seancribbs@REDACTED Tue Mar 15 13:58:14 2011 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 15 Mar 2011 08:58:14 -0400 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> References: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> Message-ID: One thing I neglected to emphasize in the announcement was that where Neotoma's parsing functions would return lists or even single characters before, they now return binaries. Some grammars will require drastic changes to accommodate this difference. My apologies to those who tried the new version and had difficulties (e.g. Cliff). Regarding Tony's question about future performance improvements, I think the next one is obvious: Although switching to binaries helped reduce copying, it can be reduced further by refusing to memoize the input in its various states of consumption, but simply to scan and split the single input binary when fetching a memoization (h/t Ben Black). This will require adding another counter to the "Index" tuple that keeps the raw byte-offset. I also want to investigate using the process dictionary a bit more instead of ETS, or at least get a comparison metric of the two approaches. I've also been thinking about what should happen for Neotoma 2.0, although as you can see by the release schedule over the last year, it has not been high enough on my priorities. Nevertheless, here's my wishlist: 1) Generate an AST from the metagrammar/grammar parser instead of raw code, decoupling parsing from code generation. 2) Create some EQC generators that can use the AST of your grammar to provide automated parser tests. 3) Allow code generation into multiple languages from the AST. One could generate Erlang, LFE, Reia, Efene, or any language for that matter. I'm thinking this is what I REALLY wanted to do when I experimented with parse_transform early on. 4) Let the grammar designer select whether to use binaries or lists as output of parsing functions. 5) Create analysis tools for the grammar AST that can discover problems and suggest optimizations. 6) Create a "regular expression" terminal in the metagrammar that can more succinctly capture strings which currently require combinations of string literals, character classes and repetition operators. Thanks for using Neotoma, and as always I'll gladly accept patches to implement any of the above. Cheers, Sean From seancribbs@REDACTED Tue Mar 15 14:06:54 2011 From: seancribbs@REDACTED (Sean Cribbs) Date: Tue, 15 Mar 2011 09:06:54 -0400 Subject: Performance Tests and Tuning In-Reply-To: References: Message-ID: Graeme, Maybe it's just me, but it looks like the largest chunk of time is spent in garbage collection, thus the natural way to optimize that is to make fewer copies of things. Also, it's natural that your list version of the Index counter is slightly slower, tuples have faster access since they are of fixed length. Sean On Tue, Mar 15, 2011 at 3:36 AM, Graeme Defty wrote: > Sorry all. This is likely to be a biggish post (and may well turn out to be > unreadable, if the tables dont reproduce well ;-) > > I would certainly appreciate someone commenting on the logic of my > approach. > > OK - heres what I have done so far, using the Reia PEG as a test case.... > > It seemed that fprof was the best tool to use, but it measures elapsed > time, not CPU. I could not find a tool to give me processor time used. > :-( (Is that really so hard to do in a VM?!?) > > As a baseline, I used the last version of the memory branch that I posted. > The top few lines of the result were as follows: > > "<0.39.0>" > > 1940718 undefined 52034.028 > > > 13.2247 > > > Count Tot Own Unit %ageTotal > > neotoma_parse p_all 4 138932 72683.399 3934.601 0.02832 7.56% > 1.0000 neotoma_parse p_attempt 4 130960 69777.613 3884.261 0.02966 7.46% > 0.9872 neotoma_parse p 5 51709 73383.129 3834.867 0.07416 7.37% > 0.9747 neotoma_parse get_memo 1 51709 5872.097 2540.128 0.04912 4.88% > 0.6456 lists foldl 3 95022 5179.214 2277.290 0.02397 4.38% > 0.5788 neotoma_parse -p_charclass/1-fun-0-' 4 14558 6096.801 2265.258 > 0.15560 4.35% > 0.5757 proplists lookup 2 137078 3229.624 2216.337 0.01617 4.26% > 0.5633 neotoma_parse -p_string/1-fun-0-' 4 110587 4273.063 2200.807 > 0.01990 4.23% > 0.5593 neotoma_parse p_advance_index 2 95022 10534.828 2080.734 0.02190 > 4.00% > 0.5288 neotoma_parse memoize 2 42376 4736.719 1954.612 0.04613 3.76% > 0.4968 neotoma_parse p_scan 4 53795 72608.579 1629.444 0.03029 3.13% > 0.4141 unicode characters_to_list 1 47433 3260.534 1588.838 0.03350 3.05% > 0.4038 neotoma_parse -p_not/1-fun-0-' 3 37517 3341.361 1537.576 0.04098 > 2.95% > 0.3908 neotoma_parse -p_anything/0-fun-0-' 2 34785 9531.222 1458.524 > 0.04193 2.80% > 0.3707 neotoma_parse memo_table_name 0 95476 1899.382 1354.767 0.01419 > 2.60% > 0.3443 proplists is_defined 2 74994 1764.019 1212.677 0.01617 2.33% > 0.3082 garbage_collect 537 > 1 1039.164 1039.164 1039.16400 2.00% > 0.2641 re run 3 14558 1056.997 994.229 0.06829 1.91% > 0.2527 ets lookup 2 52794 1155.080 950.885 0.01801 1.83% > 0.2417 ets insert 2 42676 1240.016 877.454 0.02056 1.69% > 0.2230 > For those unfamiliar with fprof, 'count' is the number of times a function > is executed, Tot is the Total time spent in a function, including time in > the called functions, and Own is the time spent in a function EXcluding > called functions (the bit in which we are most interested) > > The figures on the top line level with <0,39,0> are the totals for the > whole process. > > The 'unit' column I added, and shows the 'own' time for a function divided > by the number of times it is executed (i.e. the unit time for an execution > ;-) ) and the % of total column shows the Own time as a percentage of the > total own time. > > The bad news, as can be seen from this %age column, is that the is no > particular 'hotspot', so no obvious place to focus our efforts. > > SO . . . I just started at the top of the list. The function p_all seemed > the biggest user, but looking at the code, t is not easy to see how to make > it any slicker. The same is true of p_attempt and p. > > The first line of interest is that for get_memo. I have wondered for a > while about whether it would be better to use a proplist to store the > caching information, so that was my first experiment. My initial try showed > an improvement of about 50% reduction in time, then I remembered this was > elapsed time, and so highly sensitive to the resources being used on my > machine at the time. > > To combat this variable, the best I could think of was to scale everything > relative to the p_all time (which I did not expect to be improving any). > This is what you see in the far right column, where the own time for each > function is shown as a fraction of the p_all time. > > *Memoisation* > Having changed the memoisation routines to use a proplist, I got the > following results: > "<0.39.0>" > > 1940528 undefined 26105.719 > 50.17% > 11.9094 > > > Count Tot Own Unit %ageTotal > > neotoma_parse p_all 4 138932 49375.193 2192.023 0.01578 8.40% > 1.0000 neotoma_parse p 5 51709 49669.187 1948.556 0.03768 7.46% > 0.8889 neotoma_parse p_attempt 4 130960 47517.007 1896.901 0.01448 7.27% > 0.8654 neotoma_parse get_memo 1 51709 4467.727 1271.97 0.02460 4.87% > 0.5803 neotoma_parse memoize 2 42376 3598.339 1195.318 0.02821 4.58% > 0.5453 neotoma_parse p_advance_index 2 95022 7174.674 1161.685 0.01223 > 4.45% > 0.5300 lists foldl 3 95022 3465.169 1090.941 0.01148 4.18% > 0.4977 neotoma_parse -p_string/1-fun-0-' 4 110587 2499.068 1051.826 > 0.00951 4.03% > 0.4798 neotoma_parse p_scan 4 53795 49261.369 957.463 0.01780 3.67% > 0.4368 proplists lookup 2 137078 2055.560 953.497 0.00696 3.65% > 0.4350 re run 3 14558 1030.919 844.034 0.05798 3.23% > 0.3850 unicode characters_to_list 1 47433 2274.426 708.93 0.01495 2.72% > 0.3234 neotoma_parse memo_table_name 0 95476 1422.423 676.268 0.00708 > 2.59% > 0.3085 garbage_collect 537 > 1 674.150 669.530 1.00000 2.56% > 0.3054 neotoma_parse -p_not/1-fun-0-' 3 37517 2014.170 630.088 0.01679 > 2.41% > 0.2874 ets lookup 2 52794 1037.220 611.673 0.01159 2.34% > 0.2790 proplists is_defined 2 74994 1209.329 579.755 0.00773 2.22% > 0.2645 ets insert 2 42676 936.285 577.493 0.01353 2.21% > 0.2635 neotoma_parse -p_charclass/1-fun-0-' 4 14558 3818.591 564.022 > 0.03874 2.16% > 0.2573 neotoma_parse -p_anything/0-fun-0-' 2 34785 6316.239 489.329 > 0.01407 1.87% > 0.2232 > > As can be seen, there is some improvement in get_memo (down from 0.6456 to > 0.5803), sadly there is a corresponding INcrease in memoize (up from 0.4968 > to 0.5453) which reduces the gain to just 0.05 p_all execution-equivalents > (if you will excuse my somewhat clumsy scale). Actually, the impacts are > complex, because all sorts of other things have shifted, so I concentrated > on the figure at the top which shows the total elapsed time in terms of > p_all executions. This has reduced from 13.225 to 11.909, a reduction of > about 10%. Not stunning, but a worthwhile improvement. > > The next obvious target was advance_index. It seemed to me that the index > structure was somewhat over-engineered, so I tried replacing it with a > simple dotted pair ([x|y]). HAS to produce some benefits, right? > > Well the results were as follows: > > "<0.39.0>" > > 1940530 undefined 26095.127 > 50.15% > 13.4288 > > > Count Tot Own Unit %ageTotal > > neotoma_parse p_all 4 138932 51173.965 1943.224 0.01399 7.45% > 1.0000 neotoma_parse p 5 51709 51657.472 1923.512 0.03720 7.37% > 0.9899 neotoma_parse p_attempt 4 130960 49330.829 1855.634 0.01417 7.11% > 0.9549 neotoma_parse get_memo 1 51709 4568.310 1265.543 0.02447 4.85% > 0.6513 neotoma_parse p_advance_index 2 95022 7860.237 1165.753 0.01227 > 4.47% > 0.5999 proplists lookup 2 137078 2279.512 1160.885 0.00847 4.45% > 0.5974 lists foldl 3 95022 3769.014 1107.37 0.01165 4.24% > 0.5699 neotoma_parse -p_string/1-fun-0-' 4 110587 2909.453 1047.395 > 0.00947 4.01% > 0.5390 neotoma_parse memoize 2 42376 3365.578 962.101 0.02270 3.69% > 0.4951 neotoma_parse p_scan 4 53795 51087.806 910.642 0.01693 3.49% > 0.4686 re run 3 14558 1021.338 885.071 0.06080 3.39% > 0.4555 neotoma_parse -p_charclass/1-fun-0-' 4 14558 4297.158 766.53 > 0.05265 2.94% > 0.3945 garbage_collect 537 > 1 738.733 732.901 1.00000 2.81% > 0.3772 unicode characters_to_list 1 47433 2487.362 723.41 0.01525 2.77% > 0.3723 neotoma_parse memo_table_name 0 95476 1529.930 663.84 0.00695 > 2.54% > 0.3416 ets lookup 2 52794 1100.060 629.197 0.01192 2.41% > 0.3238 proplists is_defined 2 74994 1355.632 606.665 0.00809 2.32% > 0.3122 ets insert 2 42676 876.211 588.44 0.01379 2.25% > 0.3028 neotoma_parse -p_not/1-fun-0-' 3 37517 2047.480 547.493 0.01459 > 2.10% > 0.2817 neotoma_parse -p_anything/0-fun-0-' 2 34785 6542.401 545.422 > 0.01568 2.09% > 0.2807 > So the total has gone from 13.2247 to 13.4288. An INCREASE of about 1.5% ! > ! ! > > Hmmm - Well maybe dotted pair handling is not so great after all. I will > try a 2-item list next. > > But you can see that this does not look like it is going to be easy. One > way to improve things is to tighten up the structure of the PEG. I am sure I > could simplify it, especially around the definition of the lowest level > elements, and thus reduce the number of times all these key functions are > executed (p_all, p, etyc.) but that seems a bit like putting the cart > before the horse. I do not really want to bend my PEG (if you will excuse > the expression) for good performance, but of course if that is what it takes > . . . > > All ideas gratefully received. > > g > _____________________________________________ > On 15 March 2011 03:56, Tony Arcieri wrote: > >> Any plans for further performance improvements in the roadmap? :) >> >> >> On Sun, Mar 13, 2011 at 4:25 PM, Sean Cribbs wrote: >> >>> It's been a year coming, so I'm proud to announce a new version of >>> Neotoma, the PEG/Packrat toolkit and parser-generator. The primary >>> feature of version 1.5 is that it uses significantly less memory and >>> also supports UTF8 (via binaries), thanks to some awesome work by >>> Graeme Defty, who has also converted Reia to use a PEG. >>> >>> You can get the latest source on Github: >>> https://github.com/seancribbs/neotoma >>> >>> Neotoma is rebar-friendly as well, so adding it to a rebar project is >>> a simple deps declaration like so: >>> >>> {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, >>> "1.5"}}} >>> >>> Hope to see some of you at Erlang Factory SF next week. >>> >>> Cheers, >>> >>> Sean Cribbs >>> >> >> >> >> -- >> Tony Arcieri >> Medioh! Kudelski >> > > From ericbmerritt@REDACTED Tue Mar 15 14:34:01 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 15 Mar 2011 09:34:01 -0400 Subject: Type Specifications for Parameterized Modules Message-ID: Hello All, Is there any way to specify the types for parameters in parameterized modules? Parameterized modules are still experimental so perhaps they are not supported for specs. In any case, I have yet to find anything mentioning parameterized modules and specs in the same context. Thanks, Eric From attila.r.nohl@REDACTED Tue Mar 15 15:32:05 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 15 Mar 2011 15:32:05 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: Hello! Unfortunately the calendar:* functions crash on wrong input: 1> calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}). ** exception error: no true branch found when evaluating an if expression in function calendar:date_to_gregorian_days/3 in call from calendar:datetime_to_gregorian_seconds/1 So if the input cannot be trusted, there's still some validation to be done. 2011/3/15, Harsh J : > Hello, > > How about handling this with seconds since year 0? > calendary:datetime_to_gregorian_seconds to get seconds, adding up this > integer in seconds as you need and converting back to datetime with > calendary:gregorian_seconds_to_datetime. > > On Tue, Mar 15, 2011 at 4:24 PM, Loris Fichera > wrote: >> Hello everybody, >> >> Say I want to write a function that takes as input a timestamp (in the >> form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by >> adding a certain amount of {hours, minutes, seconds} to the input. > > -- > Harsh J > http://harshj.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From torben.lehoff@REDACTED Tue Mar 15 15:36:31 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 15 Mar 2011 15:36:31 +0100 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: References: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> Message-ID: Hi Sean, Perhaps 1.5 should have been 2.0? See http://semver.org/ where rule 9 states that: Major version X (X.y.z | X > 0) MUST be incremented if any backwards > incompatible changes are introduced to the public API. It MAY include minor > and patch level changes. > Just a thought. Cheers, Torben On Tue, Mar 15, 2011 at 13:58, Sean Cribbs wrote: > One thing I neglected to emphasize in the announcement was that where > Neotoma's parsing functions would return lists or even single > characters before, they now return binaries. Some grammars will > require drastic changes to accommodate this difference. My apologies > to those who tried the new version and had difficulties (e.g. Cliff). > > Regarding Tony's question about future performance improvements, I > think the next one is obvious: Although switching to binaries helped > reduce copying, it can be reduced further by refusing to memoize the > input in its various states of consumption, but simply to scan and > split the single input binary when fetching a memoization (h/t Ben > Black). This will require adding another counter to the "Index" tuple > that keeps the raw byte-offset. I also want to investigate using the > process dictionary a bit more instead of ETS, or at least get a > comparison metric of the two approaches. > > I've also been thinking about what should happen for Neotoma 2.0, > although as you can see by the release schedule over the last year, it > has not been high enough on my priorities. Nevertheless, here's my > wishlist: > > 1) Generate an AST from the metagrammar/grammar parser instead of raw > code, decoupling parsing from code generation. > 2) Create some EQC generators that can use the AST of your grammar to > provide automated parser tests. > 3) Allow code generation into multiple languages from the AST. One > could generate Erlang, LFE, Reia, Efene, or any language for that > matter. I'm thinking this is what I REALLY wanted to do when I > experimented with parse_transform early on. > 4) Let the grammar designer select whether to use binaries or lists as > output of parsing functions. > 5) Create analysis tools for the grammar AST that can discover > problems and suggest optimizations. > 6) Create a "regular expression" terminal in the metagrammar that can > more succinctly capture strings which currently require combinations > of string literals, character classes and repetition operators. > > Thanks for using Neotoma, and as always I'll gladly accept patches to > implement any of the above. > > Cheers, > > Sean > -- http://www.linkedin.com/in/torbenhoffmann From mazen.harake@REDACTED Tue Mar 15 15:50:34 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 15 Mar 2011 15:50:34 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: On 15 March 2011 15:32, Attila Rajmund Nohl wrote: > Hello! > > Unfortunately the calendar:* functions crash on wrong input: I would say that this is a _good_ thing. More functions in standard applications should crash when not used properly. From rtrlists@REDACTED Tue Mar 15 15:56:59 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 15 Mar 2011 14:56:59 +0000 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: Hi Attila, On Tue, Mar 15, 2011 at 2:32 PM, Attila Rajmund Nohl < attila.r.nohl@REDACTED> wrote: > Hello! > > Unfortunately the calendar:* functions crash on wrong input: > > 1> calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}). > ** exception error: no true branch found when evaluating an if expression > in function calendar:date_to_gregorian_days/3 > in call from calendar:datetime_to_gregorian_seconds/1 > > So if the input cannot be trusted, there's still some validation to be > done. > > If you want to convert the exception into a return value: 1> try calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}) catch _:_ -> bad_datetime end. bad_datetime Robby From alain.odea@REDACTED Tue Mar 15 16:15:51 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 15 Mar 2011 12:45:51 -0230 Subject: [erlang-questions] Type Specifications for Parameterized Modules In-Reply-To: References: Message-ID: Hi Eric: On Tue, Mar 15, 2011 at 11:04 AM, Eric Merritt wrote: > Hello All, > > ? Is there any way to specify the types for parameters in > parameterized modules? > > Parameterized modules are still experimental so perhaps they are not > supported for specs. In any case, I have yet to find anything > mentioning parameterized modules and specs in the same context. > > Thanks, > ?Eric > In an interview with Erlang Inside, Kostis Sagonas says parameterized modules are supported by Dialyzer: http://erlanginside.com/interview-with-kostis-sagonas-leader-of-the-hipe-team-and-erlang-tool-developer-244 However, I am not sure how the support works. From loris.fichera@REDACTED Tue Mar 15 16:22:09 2011 From: loris.fichera@REDACTED (Loris Fichera) Date: Tue, 15 Mar 2011 16:22:09 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: <4D7F8421.70300@gmail.com> On 03/15/2011 03:32 PM, Attila Rajmund Nohl wrote: > Hello! > > Unfortunately the calendar:* functions crash on wrong input: > > 1> calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}). > ** exception error: no true branch found when evaluating an if expression > in function calendar:date_to_gregorian_days/3 > in call from calendar:datetime_to_gregorian_seconds/1 > > So if the input cannot be trusted, there's still some validation to be done. Ok. Actually, in my case the input is generated via the erlang:localtime() function, so I'm sure about its validity. Loris From Ralf.Gans@REDACTED Tue Mar 15 15:53:25 2011 From: Ralf.Gans@REDACTED (Ralf P. Gans) Date: Tue, 15 Mar 2011 15:53:25 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: <06FDF1A7-967C-4E5B-B1F7-E45E41A11CD6@t-online.de> Hi, I'm very new to erlang, but isn't the exception the validation? Isn't the exception catchable? It's what I would do in C++ or Java. Or am I thinking the wrong way? Cheers, Ralf On 15.03.2011, at 15:32, Attila Rajmund Nohl wrote: > Hello! > > Unfortunately the calendar:* functions crash on wrong input: > > 1> calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}). > ** exception error: no true branch found when evaluating an if expression > in function calendar:date_to_gregorian_days/3 > in call from calendar:datetime_to_gregorian_seconds/1 > > So if the input cannot be trusted, there's still some validation to be done. > > 2011/3/15, Harsh J : >> Hello, >> >> How about handling this with seconds since year 0? >> calendary:datetime_to_gregorian_seconds to get seconds, adding up this >> integer in seconds as you need and converting back to datetime with >> calendary:gregorian_seconds_to_datetime. >> >> On Tue, Mar 15, 2011 at 4:24 PM, Loris Fichera >> wrote: >>> Hello everybody, >>> >>> Say I want to write a function that takes as input a timestamp (in the >>> form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by >>> adding a certain amount of {hours, minutes, seconds} to the input. >> >> -- >> Harsh J >> http://harshj.com >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kostis@REDACTED Tue Mar 15 16:30:06 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 15 Mar 2011 17:30:06 +0200 Subject: [erlang-questions] Type Specifications for Parameterized Modules In-Reply-To: References: Message-ID: <4D7F85FE.8050207@cs.ntua.gr> Alain O'Dea wrote: > Hi Eric: > > On Tue, Mar 15, 2011 at 11:04 AM, Eric Merritt wrote: >> Hello All, >> >> Is there any way to specify the types for parameters in >> parameterized modules? >> >> Parameterized modules are still experimental so perhaps they are not >> supported for specs. In any case, I have yet to find anything >> mentioning parameterized modules and specs in the same context. >> >> Thanks, >> Eric >> > > In an interview with Erlang Inside, Kostis Sagonas says parameterized > modules are supported by Dialyzer: > http://erlanginside.com/interview-with-kostis-sagonas-leader-of-the-hipe-team-and-erlang-tool-developer-244 > > However, I am not sure how the support works. There are two different issues here, which should not be confused: 1. Support for parameterized modules by dialyzer i.e. support for modules which do not start with -module(mod). but with: -module(mod, [Param1, ..., ParamN]). I am pretty confident that this support is in place; if you encounter any problems please report them. 2. Writing specs for functions in parameterized modules I believe that all what's needed is to modify the translation that handles parameterized modules also take the specs into account and translate them to their proper counterparts (i.e., add types for the extra function parameters). This can be as simple as adding the type any() for the extra parameters or be more fancy by extending the module declaration with the ability to specify proper types for the parameters, as in: -module(mod, [Param1 :: type1(), ..., ParamN :: typeN()]). and use these types in the translation. Volunteers wanted among the developers that care about parameterized modules. This is really really low on my todo list. Kostis From maruthavanan_s@REDACTED Tue Mar 15 17:09:01 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Tue, 15 Mar 2011 12:09:01 -0400 Subject: Optmized way to handle more timers Message-ID: Hi, I have a situation where my task can fail and I have to retry it after n seconds. The situation there may be around 100 timers in a second at the worst case. I feel creating 100 timers for this situation is not optimum. Any ideas how to handle this. Thanks, Marutha From comptekki@REDACTED Tue Mar 15 17:13:52 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 15 Mar 2011 10:13:52 -0600 Subject: counter.erl example from "Concurrent Programming in ERLANG" - debugging In-Reply-To: References: Message-ID: On Mon, Mar 14, 2011 at 3:33 PM, Wes James wrote: > I run the counter.erl in the shell - Pid=counter:start(). ?Then ran > counter:value(Pid) - returns 0. ?I wanted to see how this works in the > debugger so I run (after starting new shell or f() ): > > > c(counter,[debug_info]). > im(). > ii(counter). > iaa([init]). > > > then Pid=counter:start(). > > It then runs through some of the code lines in the debugger. ?I then > run counter:value(Pid), but then it just freezes after I press enter. > Can the debugger not debug this? > > got out of the shell then back in and just ran up to > counter:value(Pid) again and it returns 0. ?So the debugger is not > liking something. > Never mind, it works fine. The debug window was hiding behind an inactive debug window. -wes From attila.r.nohl@REDACTED Tue Mar 15 17:18:00 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 15 Mar 2011 17:18:00 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: 2011/3/15, Mazen Harake : > On 15 March 2011 15:32, Attila Rajmund Nohl wrote: >> Hello! >> >> Unfortunately the calendar:* functions crash on wrong input: > > I would say that this is a _good_ thing. More functions in standard > applications should crash when not used properly. The main problem is that the calendar module lacks a "normalizer" function (or an other version of the functions that accept "wrong" value), so now everyone writes their own "add one day to the date" function... From qwertymaniac@REDACTED Tue Mar 15 17:09:43 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Tue, 15 Mar 2011 21:39:43 +0530 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: <06FDF1A7-967C-4E5B-B1F7-E45E41A11CD6@t-online.de> References: <4D7F4567.5050801@gmail.com> <06FDF1A7-967C-4E5B-B1F7-E45E41A11CD6@t-online.de> Message-ID: Indeed, it is catchable - as Robert has demonstrated. On Tue, Mar 15, 2011 at 8:23 PM, Ralf P. Gans wrote: > Hi, > > I'm very new to erlang, but isn't the exception the validation? > Isn't the exception catchable? > > It's what I would do in C++ or Java. > > Or am I thinking the wrong way? > > Cheers, Ralf > > On 15.03.2011, at 15:32, Attila Rajmund Nohl wrote: > >> Hello! >> >> Unfortunately the calendar:* functions crash on wrong input: >> >> 1> calendar:datetime_to_gregorian_seconds({{2011,2,30},{1,2,3}}). >> ** exception error: no true branch found when evaluating an if expression >> ? ? in function ?calendar:date_to_gregorian_days/3 >> ? ? in call from calendar:datetime_to_gregorian_seconds/1 >> >> So if the input cannot be trusted, there's still some validation to be done. >> >> 2011/3/15, Harsh J : >>> Hello, >>> >>> How about handling this with seconds since year 0? >>> calendary:datetime_to_gregorian_seconds to get seconds, adding up this >>> integer in seconds as you need and converting back to datetime with >>> calendary:gregorian_seconds_to_datetime. >>> >>> On Tue, Mar 15, 2011 at 4:24 PM, Loris Fichera >>> wrote: >>>> Hello everybody, >>>> >>>> Say I want to write a function that takes as input a timestamp (in the >>>> form {{y,m,d}, {h,m,s}} ) and returns another timestamp, obtained by >>>> adding a certain amount of {hours, minutes, seconds} to the input. >>> >>> -- >>> Harsh J >>> http://harshj.com >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Harsh J http://harshj.com From kuleshovmail@REDACTED Tue Mar 15 17:30:56 2011 From: kuleshovmail@REDACTED (shk) Date: Tue, 15 Mar 2011 09:30:56 -0700 (PDT) Subject: EPmail-0.3 released Message-ID: <1300206656394-3357082.post@n4.nabble.com> Let me announce EPmail-0.3 version. EPmail - is mail system consot smtp and pop3 server written in Erlang. What's new? In this release: * Added UIDL in pop3 server, now letters can remain at the server after reading. * Fixing some bugs in pop3 server (noop command bug, list checking arguments bug) * Looping mail message receiving (Now smtp server can receive message of any size) * Bug fixed in message parsing (Now process child don't crash in large message) * Fixed popd_fsm and smtpd_fsm command order * Add VRFY command in smtp server * Add sqlite3 storage for user data (https://github.com/alexeyr/erlang-sqlite3) * Add postgresql storage for user data (https://github.com/noss/pgsql) * Add some items in configuration file * Other little bug fix and features In next version plan: * Add ssl support * Make normal message sending (now it's work but ugly hack) Please, if you interesting in EPmail project, write me feature request in Issue (at github) Links: EPmail page - https://github.com/onlyshk/epmail EPmail source EPmail wiki page - https://github.com/onlyshk/epmail/wiki EPmail wiki EPmail issue - https://github.com/onlyshk/epmail/issues EPmail issue EPmail is open source project under Erlang public license Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/EPmail-0-3-released-tp3357082p3357082.html Sent from the Erlang Questions mailing list archive at Nabble.com. From tony.arcieri@REDACTED Tue Mar 15 17:50:31 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Tue, 15 Mar 2011 10:50:31 -0600 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> References: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> Message-ID: The Reia grammar is available here: https://github.com/tarcieri/reia/blob/peg/src/compiler/reia_parse.peg On Tue, Mar 15, 2011 at 3:33 AM, Dmitrii Dimandt wrote: > > Awesome! > > Is reia grammar available? Could make a great place to start learning from > > > It's been a year coming, so I'm proud to announce a new version of > > Neotoma, the PEG/Packrat toolkit and parser-generator. The primary > > feature of version 1.5 is that it uses significantly less memory and > > also supports UTF8 (via binaries), thanks to some awesome work by > > Graeme Defty, who has also converted Reia to use a PEG. > > > > You can get the latest source on Github: > https://github.com/seancribbs/neotoma > > > > Neotoma is rebar-friendly as well, so adding it to a rebar project is > > a simple deps declaration like so: > > > > {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, > "1.5"}}} > > > > Hope to see some of you at Erlang Factory SF next week. > > > > Cheers, > > > > Sean Cribbs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Tony Arcieri Medioh! Kudelski From robert.virding@REDACTED Tue Mar 15 18:32:02 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 15 Mar 2011 18:32:02 +0100 Subject: [erlang-questions] counter.erl example from "Concurrent Programming in ERLANG" - debugging In-Reply-To: <4EFD0DB1-03C0-4561-B741-436241D4F26A@gmail.com> References: <4EFD0DB1-03C0-4561-B741-436241D4F26A@gmail.com> Message-ID: <4D7FA292.5060501@erlang-solutions.com> Of course it's the canonical book! :-) More seriously, I don't think that Erlang has changed in such a way that examples there won't run in the current systems. It is more that new things have been added since then. One fun thing in the old book are the examples which describe hardware control problems. One change, however, is that the behaviour of link/1 has changed when trying to link a dead process. Originally link/1 was completely asynchronous and would always generate a 'noproc' exit signal from the non-existent process, irrespective of whether the process was local or on another node. Now it tries to "help" by behaving differently if the process is local. Grrr. Robert On 3/15/11 1:12 PM, Alain O'Dea wrote: > Good stuff :) > > Just making sure you weren't led to believe that is the canonical book :) > > > > On 2011-03-15, at 1:57, Wes James wrote: > >> On Mon, Mar 14, 2011 at 6:15 PM, Alain O'Dea wrote: >>> Hi Wes: >> Hi Alain: >> >>> On 2011-03-14, at 19:03, Wes James wrote: >>> >>> I run the counter.erl in the shell - Pid=counter:start(). Then ran >>> counter:value(Pid) - returns 0. I wanted to see how this works in the >>> debugger so I run (after starting new shell or f() ): >> >> >>> Out of curiosity, why are you using that >>> particular book? Erlang and its tools may have changed enough to break it's >>> examples. >> Well about March of 2010 I found Erlang and I checked the University >> library here and they had this book and I checked it out. I ordered a >> copy from amazon and hope to get a few signatures next week at the SF >> Factory :) It's old, but the basics probably still hold. There might >> be a general issue with debugging spawns/receives, etc. Hopefully >> I'll have more input by tomorrow from some folks over the sea :) >> >>> Much more current resources are available: >>> Fr?d?ric Trottier-H?bert's Learn You Some Erlang for Great Good: >>> http://learnyousomeerlang.com/ >> I think I've been there a few times. >> >>> Joe Armstrong's Programming Erlang, Software for a Concurrent World: >>> http://pragprog.com/titles/jaerlang/programming-erlang >> Ordered it a couple of weeks ago and got it last week :) >> >>> Martin Logan, Richard Carlsson and Eric Merritt's Erlang and OTP in Action: >>> http://www.manning.com/logan/ >>> >>> Francesco Cesarini and Simon Thompson's Erlang Programming: >>> http://oreilly.com/catalog/9780596518189 >> I've seen these on the erlang.org side panel and might look in to them too. >> >> thx for the tips. >> >> -wes >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From bile@REDACTED Tue Mar 15 18:53:19 2011 From: bile@REDACTED (bile@REDACTED) Date: Tue, 15 Mar 2011 13:53:19 -0400 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: References: <4D7F4567.5050801@gmail.com> Message-ID: <20110315135319.4ff8d99a@firefly> On Tue, 15 Mar 2011 15:50:34 +0100 Mazen Harake wrote: > I would say that this is a _good_ thing. More functions in standard > applications should crash when not used properly. > Shouldn't the error be related to the function at hand and not it's internals? That exception isn't very useful unless you go digging around in calendar.erl From jesper.louis.andersen@REDACTED Tue Mar 15 18:55:00 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 15 Mar 2011 18:55:00 +0100 Subject: [erlang-questions] Optmized way to handle more timers In-Reply-To: References: Message-ID: On Tue, Mar 15, 2011 at 17:09, maruthavanan s wrote: > I have a situation where my task can fail and I have to retry it after n seconds. The situation there may be around 100 timers in a second at the worst case. I feel creating 100 timers for this situation is not optimum. > > Any ideas how to handle this. With timers? :) I'd use erlang:send_after/2 or erlang:start_timer/2 to install timers. It is quite much faster than the timer module and it should easily be able to cope with many simultaneous timers. You can also cancel timers again when you know they are not necessary anymore, but do use the TimerRef to make sure you don't handle old stray timers. If you are using gen_fsm's, then they have their own timers which may come in handy for you. -- J. From mazen.harake@REDACTED Tue Mar 15 21:24:55 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 15 Mar 2011 21:24:55 +0100 Subject: [erlang-questions] add a time interval to a timestamp In-Reply-To: <20110315135319.4ff8d99a@firefly> References: <4D7F4567.5050801@gmail.com> <20110315135319.4ff8d99a@firefly> Message-ID: If you use my foo/1 function wrong, say you use a string instead of an int I'd say "your problem" rather than being "nice" and returning {error, not_int}. In this case you are using an API, you should know how to call it. If you don't call it correctly it will (at least should, in some cases) crash but _why_ doesn't matter to you as long as it doesn't crash when you call it correctly, you'll never jump into an unknown *.erl file just because you called it incorrectly. I think this is a very healthy way to look at systems, makes my code much cleaner than handling {error, you_called_this_function_wrong}. PS: The exception is when {error, something} is part of the behaviour of the function E.g. {error, unable_to_connect}. My 2 cents. :) On 15 March 2011 18:53, wrote: > On Tue, 15 Mar 2011 15:50:34 +0100 > Mazen Harake wrote: > >> I would say that this is a _good_ thing. More functions in standard >> applications should crash when not used properly. >> > > Shouldn't the error be related to the function at hand and not it's > internals? That exception isn't very useful unless you go digging > around in calendar.erl > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From comptekki@REDACTED Tue Mar 15 21:57:54 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 15 Mar 2011 14:57:54 -0600 Subject: register BIF Message-ID: If I do: Pid=timer2:timeout(0,0). register(blah, Pid). I get: ** exception error: bad argument in function register/2 called as register(blah,<0.33.0>) If I do: register(blah, timer2:timeout(0,0)). it returns "true", but if I do: whereis(blah). then it returns undefined. How does register() work? thx, -wes From comptekki@REDACTED Tue Mar 15 22:06:42 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 15 Mar 2011 15:06:42 -0600 Subject: register BIF In-Reply-To: References: Message-ID: On Tue, Mar 15, 2011 at 2:57 PM, Wes James wrote: > If I do: > > Pid=timer2:timeout(0,0). > > register(blah, Pid). > > I get: > > ** exception error: bad argument > ? ? in function ?register/2 > ? ? ? ?called as register(blah,<0.33.0>) > I hate it when I find an answer right after asking the list. This: http://www.erlang-solutions.com/thesis/erlang.html pointed to the idea that in a spawn() there is spawn(arg1, arg1, arg3). I did: register(timer2, timer2:timeout(0,0)) and it worked. The first timer2 is the module name. I guess it wants the module name to register, not just any atom. -wes From fritchie@REDACTED Tue Mar 15 23:10:23 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 15 Mar 2011 17:10:23 -0500 Subject: [erlang-questions] Re: register BIF In-Reply-To: Message of "Tue, 15 Mar 2011 15:06:42 MDT." Message-ID: <6687.1300227023@snookles.snookles.com> Wes James wrote: wj> register(timer2, timer2:timeout(0,0)) and it worked. The first wj> timer2 is the module name. I guess it wants the module name to wj> register, not just any atom. Wes, that's incorrect: register/2's first arg can be any atom. But you can't register the same name twice ... even if it's the same process that is trying the second time. 1> code:which(file). "/usr/local/erlang/R14B01.64bit/lib/erlang/lib/kernel-2.14.2/ebin/file.beam" 2> code:which(asdlkfklasdfljkasdfjlkasdf). non_existing 3> self(). <0.32.0> 4> register(asdlkfklasdfljkasdfjlkasdf, self()). true 5> self(). <0.32.0> 6> register(asdlkfklasdfljkasdfjlkasdf, self()). ** exception error: bad argument in function register/2 called as register(asdlkfklasdfljkasdfjlkasdf,<0.32.0>) -Scott From comptekki@REDACTED Wed Mar 16 05:04:17 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 15 Mar 2011 22:04:17 -0600 Subject: process communication Message-ID: >From this site: http://undefined.org/c4-1/slides/slides.html I did this: erl -name bob@REDACTED -cookie secret 1> register(adder, c4_adder:start()). true 2> c4_adder:add(10, adder). {ok,10} $ erl -name alice@REDACTED -cookie secret line line copied from page -> 1> c4_adder:add(10, {adder, 'bob@REDACTED'}). line line copied from page -> {ok,20} but when I do the last line, I get: (alice@REDACTED)1> c4_adder:add(10, {adder, 'bob@REDACTED'}). {error,timeout} when I do regs(). I see nothing. If I do nregs(). I see adder. Why is alice not talking to bob? thx, -wes From comptekki@REDACTED Wed Mar 16 05:34:05 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 15 Mar 2011 22:34:05 -0600 Subject: process communication In-Reply-To: References: Message-ID: On Tue, Mar 15, 2011 at 10:04 PM, Wes James wrote: > From this site: > > http://undefined.org/c4-1/slides/slides.html > just par for today..... closed out of the erl shell on both ends..... started back up and put in the same lines and it works now. Go figure. thx, -wes From sedrik@REDACTED Wed Mar 16 09:09:15 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Wed, 16 Mar 2011 09:09:15 +0100 Subject: [erlang-questions] Optmized way to handle more timers In-Reply-To: References: Message-ID: On Tue, Mar 15, 2011 at 6:55 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Mar 15, 2011 at 17:09, maruthavanan s > wrote: > > > I have a situation where my task can fail and I have to retry it after n > seconds. The situation there may be around 100 timers in a second at the > worst case. I feel creating 100 timers for this situation is not optimum. > > > > Any ideas how to handle this. > > With timers? :) > > I'd use erlang:send_after/2 or erlang:start_timer/2 to install timers. > It is quite much faster than the timer module and it should easily be > able to cope with many simultaneous timers. You can also cancel timers > again when you know they are not necessary anymore, but do use the > TimerRef to make sure you don't handle old stray timers. If you are > using gen_fsm's, then they have their own timers which may come in > handy for you. > > For more information the *Efficiency Guide** User's Guide (see* http://erlang.org/doc/efficiency_guide/commoncaveats.html#id60654). =) > > > -- > J. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From corticalcomputer@REDACTED Wed Mar 16 10:30:13 2011 From: corticalcomputer@REDACTED (G.S.) Date: Wed, 16 Mar 2011 05:30:13 -0400 Subject: Is the digraph module the only way to deal with graphs in Erlang? Message-ID: Hello, Are there any other modules which are similar to the Digraph module? I'm working with large graphs, and though I have my own encoding system for them, I was hoping to switch to something standard, which is why I originally wanted to start using Erlang's digraph module... but it's somewhat clumsy, especially with the way it uses ETS. If anyone knows of another graph type module in Erlang, or have other suggestions, that would be great. Regards, -Gene From corticalcomputer@REDACTED Wed Mar 16 10:55:48 2011 From: corticalcomputer@REDACTED (G.S.) Date: Wed, 16 Mar 2011 05:55:48 -0400 Subject: [erlang-questions] Is the digraph module the only way to deal with graphs in Erlang? In-Reply-To: References: Message-ID: Hello Ivan, Thanks, Phoebus looks like an awesome project, and the billions of vertices is in the range of what I was looking for. But I was hoping to find something more of a just graph storage and processing (adding edges, vertices...) something closer to digraph, rather than something as vast as Phoebus. Regards, -Gene On Wed, Mar 16, 2011 at 5:37 AM, Ivan Uemlianin wrote: > How big are your graphs? Would Phoebus be appropriate? > > https://github.com/xslogic/phoebus > > There are various blog posts about it around and about. I haven't used it > myself. > > Best wishes > > Ivan > > > On 16 Mar 2011, at 09:30, G.S. wrote: > > > Hello, > > > > Are there any other modules which are similar to the Digraph module? I'm > > working with large graphs, and though I have my own encoding system for > > them, I was hoping to switch to something standard, which is why I > > originally wanted to start using Erlang's digraph module... but it's > > somewhat clumsy, especially with the way it uses ETS. > > > > If anyone knows of another graph type module in Erlang, or have other > > suggestions, that would be great. > > Regards, > > -Gene > > ============================================================ > Ivan A. Uemlianin > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > www.linkedin.com/in/ivanuemlianin > > "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > (Schiller, Beethoven) > ============================================================ > > > From jocke@REDACTED Wed Mar 16 11:32:42 2011 From: jocke@REDACTED (Joakim G.) Date: Wed, 16 Mar 2011 11:32:42 +0100 Subject: xmerl's error handling s?cks Message-ID: <4D8091CA.8020708@tail-f.com> I use xmerl to parse an XML file according to an XML schema. It works like a charm but the error handling was a painful experience indeed. I use the following: ==== load_xml(SchemaFilename, ConfigFilename) -> case catch xmerl_scan:file(ConfigFilename, [{quiet, true}, {namespace_conformant, true}, {validation, schema}, {schemaLocation, [{"foo", SchemaFilename}]}]) of {'EXIT', {fatal, Error}} -> Error; {ValidElement, []} -> {ok, ValidElement} end. ==== Then I wrote format_error/1 to take care of the errors: ==== format_error( {{failed_schema_validation, [{_, _, {undefined, {internal_error, {{badmatch, #xsd_state{errors = [{_ , _, {value_not_valid, Value, _SimpleTypes}}|_]}}, _StackTrace}}}}]}, {file, Filename}, {line, Line}, {col, Col}}) -> io_lib:format("~s: ~w: ~w: bad value: ~s", [Filename, Line, Col, Value]); format_error({{failed_schema_validation, [{type, ErrorCause, Value}|_]}, {file, Filename}, {line, Line}, {col, Col}}) -> io_lib:format("~s: ~w: ~w: ~w: ~s", [Filename, Line, Col, ErrorCause, Value]); format_error({{failed_schema_validation, [{_, _, Error}|_]}, {file, Filename}, {line, Line}, {col, Col}}) -> io_lib:format("~s: ~w: ~w: ~s", [Filename, Line, Col, lists:flatten(xmerl_xsd:format_error(Error))]); format_error({Error, {file, Filename}, {line, Line}, {col, Col}}) -> io_lib:format("~s: ~w: ~w: syntax error: ~p", [Filename, Line, Col, Error] ==== It still hurts figuring this out the hard way. I probably do something wrong. Cheers /Jocke From ivan@REDACTED Wed Mar 16 10:37:48 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 16 Mar 2011 09:37:48 +0000 Subject: [erlang-questions] Is the digraph module the only way to deal with graphs in Erlang? In-Reply-To: References: Message-ID: How big are your graphs? Would Phoebus be appropriate? https://github.com/xslogic/phoebus There are various blog posts about it around and about. I haven't used it myself. Best wishes Ivan On 16 Mar 2011, at 09:30, G.S. wrote: > Hello, > > Are there any other modules which are similar to the Digraph module? I'm > working with large graphs, and though I have my own encoding system for > them, I was hoping to switch to something standard, which is why I > originally wanted to start using Erlang's digraph module... but it's > somewhat clumsy, especially with the way it uses ETS. > > If anyone knows of another graph type module in Erlang, or have other > suggestions, that would be great. > Regards, > -Gene ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From dmitrii@REDACTED Wed Mar 16 11:50:42 2011 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Wed, 16 Mar 2011 12:50:42 +0200 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: References: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> Message-ID: <3A4B999D-D604-42DD-A3C9-29035E142B52@dmitriid.com> Thank you! > The Reia grammar is available here: > > https://github.com/tarcieri/reia/blob/peg/src/compiler/reia_parse.peg > > On Tue, Mar 15, 2011 at 3:33 AM, Dmitrii Dimandt wrote: > > Awesome! > > Is reia grammar available? Could make a great place to start learning from > > > It's been a year coming, so I'm proud to announce a new version of > > Neotoma, the PEG/Packrat toolkit and parser-generator. The primary > > feature of version 1.5 is that it uses significantly less memory and > > also supports UTF8 (via binaries), thanks to some awesome work by > > Graeme Defty, who has also converted Reia to use a PEG. > > > > You can get the latest source on Github: https://github.com/seancribbs/neotoma > > > > Neotoma is rebar-friendly as well, so adding it to a rebar project is > > a simple deps declaration like so: > > > > {neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, "1.5"}}} > > > > Hope to see some of you at Erlang Factory SF next week. > > > > Cheers, > > > > Sean Cribbs > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > -- > Tony Arcieri > Medioh! Kudelski From yujiangw@REDACTED Wed Mar 16 12:28:21 2011 From: yujiangw@REDACTED (Edward Wang) Date: Wed, 16 Mar 2011 19:28:21 +0800 Subject: Fwd: Re: [erlang-questions] Is the digraph module the only way to deal with graphs in Erlang? In-Reply-To: References: Message-ID: Cc the list. ---------- Forwarded message ---------- From: "Edward Wang" Date: Mar 16, 2011 7:27 PM Subject: Re: [erlang-questions] Is the digraph module the only way to deal with graphs in Erlang? To: "G.S." Using neo4j and its Erlang binding? I haven't use it myself. You've be warned :) http://wiki.neo4j.org/content/Main_Page Edward. On Mar 16, 2011 5:33 PM, "G.S." wrote: > Hello, > > Are there any other modules which are similar to the Digraph module? I'm > working with large graphs, and though I have my own encoding system for > them, I was hoping to switch to something standard, which is why I > originally wanted to start using Erlang's digraph module... but it's > somewhat clumsy, especially with the way it uses ETS. > > If anyone knows of another graph type module in Erlang, or have other > suggestions, that would be great. > Regards, > -Gene From cbenac@REDACTED Wed Mar 16 13:38:07 2011 From: cbenac@REDACTED (Clara Benac Earle) Date: Wed, 16 Mar 2011 13:38:07 +0100 Subject: [ANN] 2nd Madrid Erlounge 30th March Message-ID: <4D80AF2F.1080803@fi.upm.es> Hi everyone, Following the success of the 1st Madrid Erlounge, the 2nd Madrid Erlounge (or Spring Madrid Erlounge) will take place on the *30th of March* at the Facultad de Inform?tica of the Universidad Complutense de Madrid (UCM). Date Wednesday 30th of March 2011 Time 18:30 - 20:30 (talk starts at 19:00) Venue Sala de Grados (1st floor of the Facultad de Inform?tica of the Universidad Complutense of Madrid) http://gpd.sip.ucm.es/fraguas/wflp06/campus_ucm.jpg Speaker: Juan Puig, Systems Engineer at Erlang Solutions Ltd. (http://www.erlang-solutions.com/) The abstract of the talk will be sent closer to the date of the event. The event it kindly sponsored by Erlang Solutions Ltd., the GPD group at the Universidad Complutense of Madrid (http://gpd.sip.ucm.es/), and the ProTest project (http://www.protest-project.eu/). Everybody welcomed! Clara Benac Earle Babel research group (https://babel.ls.fi.upm.es/) Universidad Polit?cnica de Madrid (UPM) From kenneth.lundin@REDACTED Wed Mar 16 15:32:22 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 16 Mar 2011 15:32:22 +0100 Subject: Erlang/OTP R14B02 has been released Message-ID: Erlang/OTP R14B02 has been released. This release is mainly a stabilization of the R14B02 release (but there are some new functionality as well). You can find the README file for the release at http://www.erlang.org/download/otp_src_R14B02.readme The source distribution and binary distribution for Windows can be downloaded from http://www.erlang.org/download/otp_src_R14B02.tar.gz http://www.erlang.org/download/otp_win32_R14B02.exe The distribution can also be downloaded using the BitTorrent protocol. Use the following torrent files to download the source distribution and binary distribution for Windows: http://www.erlang.org/download/otp_src_R14B02.tar.gz.torrent http://www.erlang.org/download/otp_win32_R14B02.exe.torrent Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README file that is part of the distribution. The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R14B02.tar.gz http://www.erlang.org/download/otp_doc_man_R14B02.tar.gz We also want to thank those that sent us patches, suggestions and bug reports, The Erlang/OTP Team Kenneth Lundin, Erlang/OTP, Ericsson AB From kenneth.lundin@REDACTED Wed Mar 16 15:41:56 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 16 Mar 2011 15:41:56 +0100 Subject: Erlang/OTP R14B02 has been released In-Reply-To: References: Message-ID: Sorry I hit the send button to early. A small correction regarding the paths to the torrents: ... > > The distribution can also be downloaded using the BitTorrent > protocol. Use the following torrent files to download the source > distribution and binary distribution for Windows: > > http://www.erlang.org/download/torrents/otp_src_R14B02.tar.gz.torrent > http://www.erlang.org/download/torrents/otp_win32_R14B02.exe.torrent > ... > The Erlang/OTP Team > Kenneth Lundin, Erlang/OTP, Ericsson AB > From ulf.wiger@REDACTED Wed Mar 16 18:13:10 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 16 Mar 2011 18:13:10 +0100 Subject: first, next, prev, last in gb_trees Message-ID: <641CC0D8-4DFA-48CB-89A3-DDB11F2EB495@erlang-solutions.com> When I use ordered_set ets over gb_trees it has more than once been due to the fact that you can do wonderful stuff with first, next, prev and last - and gb_trees doesn't have them. I've made a stab at implementing these functions for the gb_trees data structure, together with a quickcheck spec to verify that they work as expected (you can use eqc mini to run the tests). I think they are reasonably efficient, but perhaps someone can think of a way to optimize them? Have at it, and pls use the spec to verify that you didn't break them* (recalling that an incorrect program can be made arbitrarily fast) * e.g. eqc:quickcheck(gb1:prop_first()) BR, Ulf W (hoping the list server won't chop the 150-line attachment) -------------- next part -------------- A non-text attachment was scrubbed... Name: gb1.erl Type: application/octet-stream Size: 2812 bytes Desc: not available URL: -------------- next part -------------- Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From john.hughes@REDACTED Wed Mar 16 20:58:52 2011 From: john.hughes@REDACTED (John Hughes) Date: Wed, 16 Mar 2011 20:58:52 +0100 Subject: [erlang-questions] first, next, prev, last in gb_trees In-Reply-To: <641CC0D8-4DFA-48CB-89A3-DDB11F2EB495@erlang-solutions.com> References: <641CC0D8-4DFA-48CB-89A3-DDB11F2EB495@erlang-solutions.com> Message-ID: <37480647E5B94C6E99257BC8027E5437@HALL> ?Nice! Slight typo: you tested prev twice... your prop_next actually tested prev, it's a copy-and-paste of prop_prev without the renaming to next! I'd suggest testing all the properties at once with eqc:module({numtests,1000},gb1). That should work in eqcmini too. One drawback of your approach is that you only test next and prev on gb_trees constructed using empty and enter. Conceivably the other functions could create gb_trees with a different structure that you might fail on. Here's some code that uses ALL of the constructors to build the test data (no bugs found though!). John %% gb_tree constructors gb() -> ?SIZED(Size, frequency([{1,{call,gb_trees,empty,[]}}, {1,{call,gb_trees,from_orddict,[orddict()]}}, {Size,?LAZY(compound_gb())}])). compound_gb() -> ?LETSHRINK([GB],[gb()], oneof([{call,gb_trees,Fun,Args++[GB]} || [Fun|Args] <- lists:map(fun tuple_to_list/1,gb_constructors())] ++ [{call,erlang,element,[3,{call,gb_trees,take_smallest,[GB]}]}, {call,erlang,element,[3,{call,gb_trees,take_largest,[GB]}]}])). gb_constructors() -> [{balance}, {delete,key()}, {delete_any,key()}, {enter,key(),val()}, {insert,key(),val()}, {update,key(),val()}]. key() -> nat(). val() -> int(). orddict() -> ?LET(L,list({key(),val()}), orddict:from_list(L)). %% Properties prop_prev2() -> ?FORALL(GB,eqc_symbolic:well_defined(gb()), begin T = eval(GB), ok == all_prev(lists:reverse(keys(T)), T) end). prop_next2() -> ?FORALL(GB,eqc_symbolic:well_defined(gb()), begin T = eval(GB), ok == all_next(keys(T), T) end). keys(T) -> [K || {K,_} <- gb_trees:to_list(T)]. ----- Original Message ----- From: "Ulf Wiger" To: "Erlang Questions" Sent: Wednesday, March 16, 2011 6:13 PM Subject: [erlang-questions] first, next, prev, last in gb_trees When I use ordered_set ets over gb_trees it has more than once been due to the fact that you can do wonderful stuff with first, next, prev and last - and gb_trees doesn't have them. I've made a stab at implementing these functions for the gb_trees data structure, together with a quickcheck spec to verify that they work as expected (you can use eqc mini to run the tests). I think they are reasonably efficient, but perhaps someone can think of a way to optimize them? Have at it, and pls use the spec to verify that you didn't break them* (recalling that an incorrect program can be made arbitrarily fast) * e.g. eqc:quickcheck(gb1:prop_first()) BR, Ulf W (hoping the list server won't chop the 150-line attachment) -------------------------------------------------------------------------------- > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > -------------------------------------------------------------------------------- > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From seancribbs@REDACTED Wed Mar 16 21:09:53 2011 From: seancribbs@REDACTED (Sean Cribbs) Date: Wed, 16 Mar 2011 16:09:53 -0400 Subject: [erlang-questions] Re: [neotoma] [ANN] Neotoma 1.5 In-Reply-To: <3A4B999D-D604-42DD-A3C9-29035E142B52@dmitriid.com> References: <8B0BC465-8FB8-4B55-93C5-E61818ED9311@dmitriid.com> <3A4B999D-D604-42DD-A3C9-29035E142B52@dmitriid.com> Message-ID: For those who aren't in like with the new binaries-by-default feature, I've branched off 1.4 and tagged 1.4.1 on github. The tag is after the rebar-izing of Neotoma, so it should still work with as a dep with your rebar-based projects. Cheers, Sean On Wed, Mar 16, 2011 at 6:50 AM, Dmitrii Dimandt wrote: > > Thank you! > > The Reia grammar is available here: > https://github.com/tarcieri/reia/blob/peg/src/compiler/reia_parse.peg > > On Tue, Mar 15, 2011 at 3:33 AM, Dmitrii Dimandt > wrote: >> >> Awesome! >> >> Is reia grammar available? Could make a great place to start learning from >> >> > It's been a year coming, so I'm proud to announce a new version of >> > Neotoma, the PEG/Packrat toolkit and parser-generator. ?The primary >> > feature of version 1.5 is that it uses significantly less memory and >> > also supports UTF8 (via binaries), thanks to some awesome work by >> > Graeme Defty, who has also converted Reia to use a PEG. >> > >> > You can get the latest source on Github: >> > https://github.com/seancribbs/neotoma >> > >> > Neotoma is rebar-friendly as well, so adding it to a rebar project is >> > a simple deps declaration like so: >> > >> > ? ?{neotoma, "1.5", {git, "git://github.com/seancribbs/neotoma", {tag, >> > "1.5"}}} >> > >> > Hope to see some of you at Erlang Factory SF next week. >> > >> > Cheers, >> > >> > Sean Cribbs >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > > -- > Tony Arcieri > Medioh! Kudelski > > From ulf.wiger@REDACTED Wed Mar 16 21:45:13 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 16 Mar 2011 21:45:13 +0100 Subject: [erlang-questions] first, next, prev, last in gb_trees In-Reply-To: <37480647E5B94C6E99257BC8027E5437@HALL> References: <641CC0D8-4DFA-48CB-89A3-DDB11F2EB495@erlang-solutions.com> <37480647E5B94C6E99257BC8027E5437@HALL> Message-ID: <4DCAF136-3AA5-443F-AA46-49CB669053FB@erlang-solutions.com> Thank you for the bugfix. :) Happy that the code was sitll correct... I figured that I would still get a decent spread of different trees, since I built them from variable-length lists of random integers - but of course, your approach is better still. BR, Ulf W On 16 Mar 2011, at 20:58, John Hughes wrote: > ?Nice! > > Slight typo: you tested prev twice... your prop_next actually tested prev, it's a copy-and-paste of prop_prev without the renaming to next! > > I'd suggest testing all the properties at once with eqc:module({numtests,1000},gb1). That should work in eqcmini too. > > One drawback of your approach is that you only test next and prev on gb_trees constructed using empty and enter. Conceivably the other functions could create gb_trees with a different structure that you might fail on. Here's some code that uses ALL of the constructors to build the test data (no bugs found though!). > > John > > %% gb_tree constructors > > gb() -> > ?SIZED(Size, > frequency([{1,{call,gb_trees,empty,[]}}, > {1,{call,gb_trees,from_orddict,[orddict()]}}, > {Size,?LAZY(compound_gb())}])). > > compound_gb() -> > ?LETSHRINK([GB],[gb()], > oneof([{call,gb_trees,Fun,Args++[GB]} > || [Fun|Args] <- > lists:map(fun tuple_to_list/1,gb_constructors())] > ++ > [{call,erlang,element,[3,{call,gb_trees,take_smallest,[GB]}]}, > {call,erlang,element,[3,{call,gb_trees,take_largest,[GB]}]}])). > > gb_constructors() -> > [{balance}, > {delete,key()}, > {delete_any,key()}, > {enter,key(),val()}, > {insert,key(),val()}, > {update,key(),val()}]. > > key() -> > nat(). > > val() -> > int(). > > orddict() -> > ?LET(L,list({key(),val()}), > orddict:from_list(L)). > > %% Properties > > prop_prev2() -> > ?FORALL(GB,eqc_symbolic:well_defined(gb()), > begin > T = eval(GB), > ok == all_prev(lists:reverse(keys(T)), T) > end). > > prop_next2() -> > ?FORALL(GB,eqc_symbolic:well_defined(gb()), > begin > T = eval(GB), > ok == all_next(keys(T), T) > end). > > keys(T) -> > [K || {K,_} <- gb_trees:to_list(T)]. > > > ----- Original Message ----- From: "Ulf Wiger" > To: "Erlang Questions" > Sent: Wednesday, March 16, 2011 6:13 PM > Subject: [erlang-questions] first, next, prev, last in gb_trees > > > When I use ordered_set ets over gb_trees it has more than once been due to the fact that you can do wonderful stuff with first, next, prev and last - and gb_trees doesn't have them. > > I've made a stab at implementing these functions for the gb_trees data structure, together with a quickcheck spec to verify that they work as expected (you can use eqc mini to run the tests). I think they are reasonably efficient, but perhaps someone can think of a way to optimize them? Have at it, and pls use the spec to verify that you didn't break them* (recalling that an incorrect program can be made arbitrarily fast) > > * e.g. eqc:quickcheck(gb1:prop_first()) > > BR, > Ulf W (hoping the list server won't chop the 150-line attachment) > > > > > -------------------------------------------------------------------------------- > > >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> > > > -------------------------------------------------------------------------------- > > >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jozsef.berces@REDACTED Thu Mar 17 04:21:59 2011 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Thu, 17 Mar 2011 04:21:59 +0100 Subject: inets httpd problems Message-ID: <3717CEF51B134A4F8073627839A9DF4120EBAE9929@ESESSCMS0353.eemea.ericsson.se> Hi, Up to OTP R14A I used the same Apache-like config file without any problem for inets httpd. Starting httpd with the same config file under R14B01, it seemed that the server did not accept any connections. After some experimenting it seems to me that the problem is with the IPv6 fallback. The default ipfamily setting is inet6fb4. With this setting the server is not accessible if I give an IPv4 address in the browser. And it seems it is not possible to change this setting. Even after inets:start(httpd,[{file,ConfFile},{ipfamily,inet}]). or inets:start(httpd,[{ipfamily,inet},{file,ConfFile}]). the ipfamily setting printed with ets:i still shows inet6fb4. And of course, the server is not accessible with IPv4 addresses. Am I doing something wrong? Thanks, Jozsef From qoocku@REDACTED Thu Mar 17 11:51:39 2011 From: qoocku@REDACTED (=?UTF-8?B?RGFtaWFuIERvYnJvY3p5xYRza2k=?=) Date: Thu, 17 Mar 2011 11:51:39 +0100 Subject: Erlang/OTP R14B02 generating documentation from source problem In-Reply-To: References: Message-ID: <4D81E7BB.8090802@gmail.com> Hi. I downloaded, compiled and tried to "make docs" and "make install-docs". The first went fine. The result of the latter is the following: escript: exception error: undefined function xmerl:export_simple/2 in function erl_eval:do_apply/5 in call from erl_eval:expr/5 in call from erl_eval:exprs/5 in call from erl_eval:local_func/5 in call from escript:interpret/4 in call from escript:start/1 in call from init:start_it/1 So, I installed without docs ("make install") and then tried again. The result was the same. So I removed all escript links and files that I had had already (some artifacts from previous releases and such) so that I had only fresh escript in the installation directory and a link in the prefix (/usr/local/bin) directory. The result was the same. Can somebody help me, please? D. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0x0CAE3AEB.asc Type: application/pgp-keys Size: 5071 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: OpenPGP digital signature URL: From lukas.larsson@REDACTED Thu Mar 17 11:59:03 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Thu, 17 Mar 2011 10:59:03 +0000 (GMT) Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem In-Reply-To: <4D81E7BB.8090802@gmail.com> Message-ID: <311465162.73841300359543943.JavaMail.root@zimbra> It seems to me like xmerl is not in the erlang install which is in your path. Check that xmerl-x.y.z is in the lib/ dir of the erlang you have in your path. or alternatively that you can load the xmerl module in a normal erlang shell. Lukas ----- Original Message ----- From: "Damian Dobroczy?ski" To: erlang-questions@REDACTED Sent: Thursday, 17 March, 2011 11:51:39 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem Hi. I downloaded, compiled and tried to "make docs" and "make install-docs". The first went fine. The result of the latter is the following: escript: exception error: undefined function xmerl:export_simple/2 in function erl_eval:do_apply/5 in call from erl_eval:expr/5 in call from erl_eval:exprs/5 in call from erl_eval:local_func/5 in call from escript:interpret/4 in call from escript:start/1 in call from init:start_it/1 So, I installed without docs ("make install") and then tried again. The result was the same. So I removed all escript links and files that I had had already (some artifacts from previous releases and such) so that I had only fresh escript in the installation directory and a link in the prefix (/usr/local/bin) directory. The result was the same. Can somebody help me, please? D. From john.hughes@REDACTED Thu Mar 17 15:54:09 2011 From: john.hughes@REDACTED (John Hughes) Date: Thu, 17 Mar 2011 15:54:09 +0100 Subject: New release of QuickCheck Message-ID: Quviq is happy to announce the release of Quviq QuickCheck 1.24.1. The two new features we are most excited by are: 1) We now have an interface to the McErlang model-checker, which makes it easy to generate random test cases and then execute them with McErlang, if you have it installed. 2) Test suite generation: QuickCheck users can now generate a FIXED test suite from a QuickCheck specification, which other people can then run without a licence. This enables open source developers, for example, to distribute QuickCheck properties together with their code, and allow any of their users in turn to test the properties in their own environment. Test suites can be generated randomly, to achieve maximum code coverage, or to cover any other user defined criteria. We believe this will provide substantial additional value. As usual, the new version can be downloaded from http://quviq-licencer.com/downloads/eqc.zip. John Hughes From bob@REDACTED Thu Mar 17 17:05:51 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 17 Mar 2011 16:05:51 +0000 Subject: Binary no-copy Message-ID: <4D82315F.5030201@bobcowdery.plus.com> Hi, Am I correct in thinking that the code fragments below will not copy any binary data. I've not tested the function yet so it may have bugs but I'm interested in whether it will be optimal in efficiency. State starts off as an empty binary. Frames are decoded and placed in a block. There are not an integral number of frames to a block. The function dsp:process_block() is in another gen-server. Thanks Bob %% Constants %% Input scale factor -define (S, (1 bsl 23) - 1). %% Block size for DSP (interleaved I and Q) -define (BlockSz, 1024). %% Samples in a Metis frame (interleaved I and Q) -define (MetisSmpls, 63). %% Bytes in a float -define (FloatSz, 4). %% Size of binary for given block size -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). do_decode_frame(Frame, State) -> %% Separate the command bytes from the frame data <> = Frame, %% What's left now is <> %% This one liner converts the I and Q to scaled floats Samples = << <<(I/?S):32/float, (Q/?S):32/float>> || <> <= Rest >>, CurrentSz = size(State), FrameSz = size(Samples), if (CurrentSz + FrameSz) < ?BinSz -> NewState = <>; true -> %% Binary part of this transfer CutAt = ?BinSz - CurrentSz - 1, Bp = binary_part(Samples, {0, CutAt}), %% Binary part of remainder Br = binary_part(Samples, {CutAt, FrameSz - CutAt}), %% Process this block dsp:process_block(<>), NewState = Br end, {ok, NewState}. From bob@REDACTED Thu Mar 17 19:31:45 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 17 Mar 2011 18:31:45 +0000 Subject: [erlang-questions] Binary no-copy In-Reply-To: References: <4D82315F.5030201@bobcowdery.plus.com> Message-ID: <4D825391.5040201@bobcowdery.plus.com> Thanks. More or less what I thought that the only copy is in the append operation. The one liner is the big one in terms of time but I will profile the whole thing eventually. I don't think your solution does quite the same thing as the sent block must be exactly the correct size not >= but I appreciate the kind of structure you are using with small clauses is more elegant. Bob On 17/03/2011 17:11, Per Gustafsson wrote: > It will copy binaries, but essentially the state binary will act like > a growable array, pre-allocating space to grow in, so there will be > some copying , but it will be linear in the size of the resulting > array. > It might be a little bit faster to rewrite this as shown below. The > reason I think this might be faster is that you do not create a > temporary samples array that just gets copied. Whether this is faster > or not in reality is of course a matter of benchmarking. > > %% Constants > %% Input scale factor > -define (S, (1 bsl 23) - 1). > %% Block size for DSP (interleaved I and Q) > -define (BlockSz, 1024). > %% Samples in a Metis frame (interleaved I and Q) > -define (MetisSmpls, 63). > %% Bytes in a float > -define (FloatSz, 4). > %% Size of binary for given block size > -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). > > do_decode_frame(Frame, State) -> > %% Separate the command bytes from the frame data > <> = Frame, > decode_samples(Rest, State). > > decode_samples(Frame, State) when size(State) >= ?BinSz -> > dsp:process_block(State), > decode_samples(Frame, <<>>); > decode_samples(< Rest/binary>>, State) -> > decode_samples(<>; > decode_samples(<<>>, State) -> > {ok, State}. > > 2011/3/17 Bob Cowdery >> Hi, >> >> Am I correct in thinking that the code fragments below will not copy any >> binary data. I've not tested the function yet so it may have bugs but >> I'm interested in whether it will be optimal in efficiency. State starts >> off as an empty binary. Frames are decoded and placed in a block. There >> are not an integral number of frames to a block. The function >> dsp:process_block() is in another gen-server. >> >> Thanks >> Bob >> >> %% Constants >> %% Input scale factor >> -define (S, (1 bsl 23) - 1). >> %% Block size for DSP (interleaved I and Q) >> -define (BlockSz, 1024). >> %% Samples in a Metis frame (interleaved I and Q) >> -define (MetisSmpls, 63). >> %% Bytes in a float >> -define (FloatSz, 4). >> %% Size of binary for given block size >> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). >> >> do_decode_frame(Frame, State) -> >> %% Separate the command bytes from the frame data >> <> = Frame, >> %% What's left now is <> >> %% This one liner converts the I and Q to scaled floats >> Samples = << <<(I/?S):32/float, (Q/?S):32/float>> || >> <> <= Rest >>, >> CurrentSz = size(State), >> FrameSz = size(Samples), >> if >> (CurrentSz + FrameSz) < ?BinSz -> >> NewState = <>; >> true -> >> %% Binary part of this transfer >> CutAt = ?BinSz - CurrentSz - 1, >> Bp = binary_part(Samples, {0, CutAt}), >> %% Binary part of remainder >> Br = binary_part(Samples, {CutAt, FrameSz - CutAt}), >> %% Process this block >> dsp:process_block(<>), >> NewState = Br >> end, >> {ok, NewState}. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> From zabrane3@REDACTED Thu Mar 17 22:02:17 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 17 Mar 2011 22:02:17 +0100 Subject: Jinterface: communication between Java and Erlang using system file Message-ID: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> Hi guys, Please, have a look to the following simple Java code which try to encode a simple Erlang tuple and save it to disk. -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 // encode a simple Erlang tuple {ok, "Hello World"} OtpErlangObject[] reply = new OtpErlangObject[2]; reply[0] = new OtpErlangAtom("ok"); reply[1] = new OtpErlangBinary("Hello World!"); OtpErlangTuple tuple = new OtpErlangTuple(reply); // create an OtpOutputStream object from the binary OtpErlangBinary binary = new OtpErlangBinary(tuple); OtpOutputStream oos = new OtpOutputStream(binary); oos.writeTo(fos); // write the Erlang binary term to a file File f = new File ("a.txt"); FileOutputStream fos = new FileOutputStream(f); fos.flush(); fos.close(); -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 When trying to read it from disk, I got an exception: $ erl 1> {ok, Bin} = file:read_file("a.txt"). {ok,<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109, 46,101,114,105,99,115,115,111,110,46,111,...>>} 2> binary_to_term(Bin). ** exception error: bad argument in function binary_to_term/1 called as binary_to_term(<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109,46, 101,114,105,99,115,115,111,110,46,111,116,...>>) What am I doing wrong? Help appreciated ... Regards, Zab From alain.odea@REDACTED Thu Mar 17 23:10:40 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Thu, 17 Mar 2011 19:40:40 -0230 Subject: [erlang-questions] Jinterface: communication between Java and Erlang using system file In-Reply-To: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> References: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> Message-ID: Hi Zabrane: On 2011-03-17, at 18:32, Zabrane Mickael wrote: > Hi guys, > > Please, have a look to the following simple Java code > which try to encode a simple Erlang tuple and save it to disk. > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > // encode a simple Erlang tuple {ok, "Hello World"} > OtpErlangObject[] reply = new OtpErlangObject[2]; > reply[0] = new OtpErlangAtom("ok"); > reply[1] = new OtpErlangBinary("Hello World!"); > OtpErlangTuple tuple = new OtpErlangTuple(reply); > > // create an OtpOutputStream object from the binary > OtpErlangBinary binary = new OtpErlangBinary(tuple); > OtpOutputStream oos = new OtpOutputStream(binary); > > oos.writeTo(fos); > > // write the Erlang binary term to a file > File f = new File ("a.txt"); > FileOutputStream fos = new FileOutputStream(f); > > fos.flush(); > fos.close(); > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > When trying to read it from disk, I got an exception: > $ erl > 1> {ok, Bin} = file:read_file("a.txt"). > {ok,<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109, > 46,101,114,105,99,115,115,111,110,46,111,...>>} > 2> binary_to_term(Bin). > ** exception error: bad argument > in function binary_to_term/1 > called as binary_to_term(<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109,46, > 101,114,105,99,115,115,111,110,46,111,116,...>>) > > > What am I doing wrong? > Help appreciated ... > > Regards, > Zab I published http://concise-software.blogspot.com/2009/08/concise-howtos-message-from-java-to.html a while back which uses node-to-node communication between Java and Erlang. It may be better suited to your problem than using files as it naturally handles the marshalling an unmarshalling if messages in both directions. From jay@REDACTED Thu Mar 17 23:51:03 2011 From: jay@REDACTED (Jay Nelson) Date: Thu, 17 Mar 2011 15:51:03 -0700 Subject: R14B02 compilation error on MacBook Pro Message-ID: Using a brand new 15" MacBook Pro Quadcore I get the following with --enable-darwin-64bit --enable-halfword-emulator: gcc -mdynamic-no-pic -m64 -g -O3 -I/Users/jay/Erlang/otp_src_R14B02/erts/i386-apple-darwin10.7.1 -no-cpp-precomp -D_XOPEN_SOURCE -DERTS_SMP -DHAVE_CONFIG_H -Wall -Wstrict-prototypes -Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -Ii386-apple-darwin10.7.1/opt/smp -Ibeam -Isys/unix -Isys/common -Ii386-apple-darwin10.7.1 -Izlib -Ipcre -Ihipe -I../include -I../include/i386-apple-darwin10.7.1 -I../include/internal -I../include/internal/i386-apple-darwin10.7.1 -c hipe/hipe_amd64.c -o obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o hipe/hipe_amd64.c:40: warning: large integer implicitly truncated to unsigned type hipe/hipe_amd64.c:43: error: conflicting types for ?hipe_patch_load_fe? hipe/hipe_arch.h:28: error: previous declaration of ?hipe_patch_load_fe? was here hipe/hipe_amd64.c:50: error: conflicting types for ?hipe_patch_insn? hipe/hipe_arch.h:29: error: previous declaration of ?hipe_patch_insn? was here hipe/hipe_amd64.c: In function ?hipe_patch_call?: hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different size hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different size hipe/hipe_amd64.c: In function ?hipe_bifs_write_u64_2?: hipe/hipe_amd64.c:371: warning: passing argument 2 of ?term_to_Uint? from incompatible pointer type make[3]: *** [obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o] Error 1 make[2]: *** [opt] Error 2 make[1]: *** [smp] Error 2 make: *** [emulator] Error 2 2 GHz Intel Core i7 14 GB 1333 MHz DDR3 Version 10.6.6 jay From anders.nygren@REDACTED Fri Mar 18 00:22:47 2011 From: anders.nygren@REDACTED (Anders Nygren) Date: Thu, 17 Mar 2011 17:22:47 -0600 Subject: [erlang-questions] R14B02 compilation error on MacBook Pro In-Reply-To: References: Message-ID: In http://www.erlang.org/cgi-bin/ezmlm-cgi?2:mss:2254:201103:glecllcjnknfjjflcikb It says that hipe does not work with the halfword-emulator. So try with --disable-hipe /Anders On Thu, Mar 17, 2011 at 4:51 PM, Jay Nelson wrote: > Using a brand new 15" MacBook Pro Quadcore I get the following with --enable-darwin-64bit --enable-halfword-emulator: > > gcc ?-mdynamic-no-pic -m64 -g -O3 -I/Users/jay/Erlang/otp_src_R14B02/erts/i386-apple-darwin10.7.1 ?-no-cpp-precomp ?-D_XOPEN_SOURCE -DERTS_SMP -DHAVE_CONFIG_H -Wall -Wstrict-prototypes -Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS ?-Ii386-apple-darwin10.7.1/opt/smp -Ibeam -Isys/unix -Isys/common -Ii386-apple-darwin10.7.1 -Izlib ?-Ipcre -Ihipe -I../include -I../include/i386-apple-darwin10.7.1 -I../include/internal -I../include/internal/i386-apple-darwin10.7.1 -c hipe/hipe_amd64.c -o obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o > hipe/hipe_amd64.c:40: warning: large integer implicitly truncated to unsigned type > hipe/hipe_amd64.c:43: error: conflicting types for ?hipe_patch_load_fe? > hipe/hipe_arch.h:28: error: previous declaration of ?hipe_patch_load_fe? was here > hipe/hipe_amd64.c:50: error: conflicting types for ?hipe_patch_insn? > hipe/hipe_arch.h:29: error: previous declaration of ?hipe_patch_insn? was here > hipe/hipe_amd64.c: In function ?hipe_patch_call?: > hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different size > hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different size > hipe/hipe_amd64.c: In function ?hipe_bifs_write_u64_2?: > hipe/hipe_amd64.c:371: warning: passing argument 2 of ?term_to_Uint? from incompatible pointer type > make[3]: *** [obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o] Error 1 > make[2]: *** [opt] Error 2 > make[1]: *** [smp] Error 2 > make: *** [emulator] Error 2 > > > 2 GHz Intel Core i7 > 14 GB 1333 MHz DDR3 > Version 10.6.6 > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From wallentin.dahlberg@REDACTED Fri Mar 18 00:25:16 2011 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Fri, 18 Mar 2011 00:25:16 +0100 Subject: [erlang-questions] R14B02 compilation error on MacBook Pro In-Reply-To: References: Message-ID: HiPE and Halfword cannot coexist, --disable-hipe // Bj?rn-Egil 2011/3/17 Jay Nelson > Using a brand new 15" MacBook Pro Quadcore I get the following with > --enable-darwin-64bit --enable-halfword-emulator: > > gcc -mdynamic-no-pic -m64 -g -O3 > -I/Users/jay/Erlang/otp_src_R14B02/erts/i386-apple-darwin10.7.1 > -no-cpp-precomp -D_XOPEN_SOURCE -DERTS_SMP -DHAVE_CONFIG_H -Wall > -Wstrict-prototypes -Wmissing-prototypes -Wdeclaration-after-statement > -DUSE_THREADS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS > -Ii386-apple-darwin10.7.1/opt/smp -Ibeam -Isys/unix -Isys/common > -Ii386-apple-darwin10.7.1 -Izlib -Ipcre -Ihipe -I../include > -I../include/i386-apple-darwin10.7.1 -I../include/internal > -I../include/internal/i386-apple-darwin10.7.1 -c hipe/hipe_amd64.c -o > obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o > hipe/hipe_amd64.c:40: warning: large integer implicitly truncated to > unsigned type > hipe/hipe_amd64.c:43: error: conflicting types for ?hipe_patch_load_fe? > hipe/hipe_arch.h:28: error: previous declaration of ?hipe_patch_load_fe? > was here > hipe/hipe_amd64.c:50: error: conflicting types for ?hipe_patch_insn? > hipe/hipe_arch.h:29: error: previous declaration of ?hipe_patch_insn? was > here > hipe/hipe_amd64.c: In function ?hipe_patch_call?: > hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different > size > hipe/hipe_amd64.c:77: warning: cast from pointer to integer of different > size > hipe/hipe_amd64.c: In function ?hipe_bifs_write_u64_2?: > hipe/hipe_amd64.c:371: warning: passing argument 2 of ?term_to_Uint? from > incompatible pointer type > make[3]: *** [obj/i386-apple-darwin10.7.1/opt/smp/hipe_amd64.o] Error 1 > make[2]: *** [opt] Error 2 > make[1]: *** [smp] Error 2 > make: *** [emulator] Error 2 > > > 2 GHz Intel Core i7 > 14 GB 1333 MHz DDR3 > Version 10.6.6 > > jay > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kaiduanx@REDACTED Fri Mar 18 02:19:26 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Thu, 17 Mar 2011 21:19:26 -0400 Subject: One minor mismatch between code and documentation for dbg:trace_client() Message-ID: Hi, all, I just found a mismatch between the code and the documentation of dbg:trace_client(). From http://www.erlang.org/doc/man/dbg.html#trace_port-2 it says, trace_port(Type, Parameters) -> fun() Type = ip | file Parameters = Filename | WrapFilesSpec | IPPortSpec IPPortSpec = PortNumber | {PortNumber, QueSize} In fact, you can specify the hostname in the IPPortSpec from the code in OTP-R14B01, trace_client(ip, Portno) when is_integer(Portno) -> trace_client1(ip, {"localhost", Portno}, {fun dhandler/2,user}); trace_client(ip, {Host, Portno}) when is_integer(Portno) -> trace_client1(ip, {Host, Portno}, {fun dhandler/2,user}) With the capability to specify host, it is very handy for remote tracing. Please correct the documentation, thanks, /Kaiduan From bpdp.erlang@REDACTED Fri Mar 18 03:49:29 2011 From: bpdp.erlang@REDACTED (Bambang Purnomosidi D. P.) Date: Fri, 18 Mar 2011 09:49:29 +0700 Subject: make docs problem in Linux (R14B02). Message-ID: Hi, I came across this error when I did "make docs". It looks like a proble in .fo file generated by xsltproc: ... ... date=`date +"%B %e %Y"`; \ xsltproc \ --stringparam docgen "/home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen" \ --stringparam gendate "$date" \ --stringparam appname "docbuilder" \ --stringparam appver "0.9.8.9" \ --xinclude \ -path /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/docbuilder_dtd \ -path /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/dtd_html_entities \ /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/xsl/db_pdf.xsl book.xml > ../pdf/docbuilder-0.9.8.9.fo fop -fo ../pdf/docbuilder-0.9.8.9.fo -pdf ../pdf/docbuilder-0.9.8.9.pdf Mar 18, 2011 9:29:26 AM org.apache.fop.events.LoggingEventListener processEvent WARNING: Line 1 of a paragraph overflows the available area by more than 50 points. (See position 2157:185) Mar 18, 2011 9:29:26 AM org.apache.fop.cli.InputHandler error SEVERE: javax.xml.transform.TransformerException: factor < 0; was: -1 Mar 18, 2011 9:29:26 AM org.apache.fop.cli.Main startFOP SEVERE: Exception javax.xml.transform.TransformerException: java.lang.IllegalArgumentException: factor < 0; was: -1 ... ... Did someone experience this too? any patches? Thanks! --bambang From boris.muehmer@REDACTED Fri Mar 18 06:36:20 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Fri, 18 Mar 2011 06:36:20 +0100 Subject: [erlang-questions] make docs problem in Linux (R14B02). In-Reply-To: References: Message-ID: Which "fop" version do You use? I think I got this error when I used 1.0... try 0.95 instead. - boris 2011/3/18 Bambang Purnomosidi D. P. : > Hi, > > I came across this error when I did "make docs". It looks like a proble in > .fo file generated by xsltproc: > > ... > ... > date=`date +"%B %e %Y"`; \ > ? ? ? ?xsltproc \ > ? ? ? ? --stringparam docgen "/home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen" > \ > ? ? ? ? --stringparam gendate "$date" \ > ? ? ? ? --stringparam appname "docbuilder" \ > ? ? ? ? --stringparam appver "0.9.8.9" \ > ? ? ? ? --xinclude ?\ > ? ? ? ? -path > /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/docbuilder_dtd \ > ? ? ? ? -path > /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/dtd_html_entities \ > ? ? ? ? ? /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/xsl/db_pdf.xsl > book.xml > ../pdf/docbuilder-0.9.8.9.fo > fop -fo ../pdf/docbuilder-0.9.8.9.fo -pdf ../pdf/docbuilder-0.9.8.9.pdf > Mar 18, 2011 9:29:26 AM org.apache.fop.events.LoggingEventListener > processEvent > WARNING: Line 1 of a paragraph overflows the available area by more than 50 > points. (See position 2157:185) > Mar 18, 2011 9:29:26 AM org.apache.fop.cli.InputHandler error > SEVERE: javax.xml.transform.TransformerException: factor < 0; was: -1 > Mar 18, 2011 9:29:26 AM org.apache.fop.cli.Main startFOP > SEVERE: Exception > javax.xml.transform.TransformerException: > java.lang.IllegalArgumentException: factor < 0; was: -1 > ... > ... > > Did someone experience this too? any patches? > > Thanks! > > --bambang > From bpdp.erlang@REDACTED Fri Mar 18 10:33:13 2011 From: bpdp.erlang@REDACTED (Bambang Purnomosidi D. P.) Date: Fri, 18 Mar 2011 16:33:13 +0700 Subject: [erlang-questions] make docs problem in Linux (R14B02). In-Reply-To: References: Message-ID: Hi Boris, Yes, I used 1.0. I changed to 0.95 and it does work like a charm. Thanks! --bambang On Fri, Mar 18, 2011 at 12:36 PM, Boris M?hmer wrote: > Which "fop" version do You use? > > I think I got this error when I used 1.0... try 0.95 instead. > > > - boris > > > 2011/3/18 Bambang Purnomosidi D. P. : > > Hi, > > > > I came across this error when I did "make docs". It looks like a proble > in > > .fo file generated by xsltproc: > > > > ... > > ... > > date=`date +"%B %e %Y"`; \ > > xsltproc \ > > --stringparam docgen > "/home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen" > > \ > > --stringparam gendate "$date" \ > > --stringparam appname "docbuilder" \ > > --stringparam appver "0.9.8.9" \ > > --xinclude \ > > -path > > /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/docbuilder_dtd \ > > -path > > /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/dtd_html_entities \ > > > /home/bpdp/tmp/otp_src_R14B02/lib/erl_docgen/priv/xsl/db_pdf.xsl > > book.xml > ../pdf/docbuilder-0.9.8.9.fo > > fop -fo ../pdf/docbuilder-0.9.8.9.fo -pdf ../pdf/docbuilder-0.9.8.9.pdf > > Mar 18, 2011 9:29:26 AM org.apache.fop.events.LoggingEventListener > > processEvent > > WARNING: Line 1 of a paragraph overflows the available area by more than > 50 > > points. (See position 2157:185) > > Mar 18, 2011 9:29:26 AM org.apache.fop.cli.InputHandler error > > SEVERE: javax.xml.transform.TransformerException: factor < 0; was: -1 > > Mar 18, 2011 9:29:26 AM org.apache.fop.cli.Main startFOP > > SEVERE: Exception > > javax.xml.transform.TransformerException: > > java.lang.IllegalArgumentException: factor < 0; was: -1 > > ... > > ... > > > > Did someone experience this too? any patches? > > > > Thanks! > > > > --bambang > > > From rtrlists@REDACTED Fri Mar 18 11:15:46 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 18 Mar 2011 10:15:46 +0000 Subject: [erlang-questions] Jinterface: communication between Java and Erlang using system file In-Reply-To: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> References: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> Message-ID: Hi Zab, On Thu, Mar 17, 2011 at 9:02 PM, Zabrane Mickael wrote: > Hi guys, > > Please, have a look to the following simple Java code > which try to encode a simple Erlang tuple and save it to disk. > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > // encode a simple Erlang tuple {ok, "Hello World"} > OtpErlangObject[] reply = new OtpErlangObject[2]; > reply[0] = new OtpErlangAtom("ok"); > reply[1] = new OtpErlangBinary("Hello World!"); > OtpErlangTuple tuple = new OtpErlangTuple(reply); > > // create an OtpOutputStream object from the binary > OtpErlangBinary binary = new OtpErlangBinary(tuple); > OtpOutputStream oos = new OtpOutputStream(binary); > > oos.writeTo(fos); > > // write the Erlang binary term to a file > File f = new File ("a.txt"); > FileOutputStream fos = new FileOutputStream(f); > > fos.flush(); > fos.close(); > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > When trying to read it from disk, I got an exception: > $ erl > 1> {ok, Bin} = file:read_file("a.txt"). > {ok,<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109, > 46,101,114,105,99,115,115,111,110,46,111,...>>} > 2> binary_to_term(Bin). > ** exception error: bad argument > in function binary_to_term/1 > called as > binary_to_term(<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109,46, > > 101,114,105,99,115,115,111,110,46,111,116,...>>) > > > What am I doing wrong? > Help appreciated ... > > Regards, > Zab > > I haven't done much Java file writing/reading for a long time, so I've forgotten a lot. But is it possible that the file gets written in "text" mode? And the IO system helpfully translates line ending characters for you? Since you're writing binary data, it's always a good idea to make sure you're writing to a binary file. If I remember correctly the Erlang file module functions assume binary by default. Not sure about Java tho'. Robby From dafgar@REDACTED Fri Mar 18 13:32:31 2011 From: dafgar@REDACTED (Sergei Fomin) Date: Fri, 18 Mar 2011 15:32:31 +0300 Subject: Driver level locking of port driver Message-ID: <4D8350DF.1090600@yandex-team.ru> Hello. The second decade of the XXI century began recently, but some linked-in drivers haven't made use of 4-cores monsters yet. :) Here is my question. Suppose I've an erlang VM (R14B02) with several scheduler threads. One of the threads enters linked-in driver which has driver level locking. At the same moment another thread wants to run driver code for a different port process. Would this thread: * be locked until the first one frees the driver; * detect the lock and re-schedule the port process; * do anything else? -- Sergei Fomin From bob@REDACTED Fri Mar 18 14:44:51 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 18 Mar 2011 13:44:51 +0000 Subject: [erlang-questions] Binary no-copy In-Reply-To: References: <4D82315F.5030201@bobcowdery.plus.com> <4D825391.5040201@bobcowdery.plus.com> Message-ID: <4D8361D3.3060202@bobcowdery.plus.com> Humour me. I'm still learning erlang and it still takes me ages to write a few lines at times. Comments in-line. Bob On 17/03/2011 20:54, Per Gustafsson wrote: > You're right it should be as below to get the same behaviour, but I > think it will work the same for input that makes sense. > Straight forward > do_decode_frame(Frame, State) -> > %% Separate the command bytes from the frame data > <> = Frame, > decode_samples(Rest, State). > First clause will match a full block where frames and blocks are aligned. Never happen in practice as 63 will never be a multiple of 64 but it's right that the code should not know this. > decode_samples(Frame, FullBlock) when bytesize(FullBlock) == ?BinSize -> > dsp:process_block(FullBlock), > decode_samples(Frame, <<>>); Second clause will match when a FullBlock has > ?BinSize and will split into ?BinSize and Rest > decode_samples(Frame, <>) -> > dsp:process_block(FullBlock), > decode_samples(Frame, Rest); Third clause. Now I get lost. This will match pretty much any binary > 8 bytes and any State. So this will get called from first and second clause. The next two clauses seem to run into each other. Don't understand that. Is the binary comprehension hidden somewhere in there? > decode_samples(< Rest/binary>>, State) -> > decode_samples(<>; > decode_samples(<<>>, State) -> > {ok, State}. > Would this work as the last two clauses: decode_samples(Frame, State) -> decode_samples(<<>>, << State/binary, <<(I/?S):32/float, (Q/?S):32/float>> || <> <= Frame >>) decode_samples(<<>>, State) -> {ok, State}. > > 2011/3/17 Bob Cowdery >: > > Thanks. More or less what I thought that the only copy is in the append > > operation. The one liner is the big one in terms of time but I will > > profile the whole thing eventually. I don't think your solution does > > quite the same thing as the sent block must be exactly the correct size > > not >= but I appreciate the kind of structure you are using with small > > clauses is more elegant. > > > > Bob > > > > On 17/03/2011 17:11, Per Gustafsson wrote: > >> It will copy binaries, but essentially the state binary will act like > >> a growable array, pre-allocating space to grow in, so there will be > >> some copying , but it will be linear in the size of the resulting > >> array. > >> It might be a little bit faster to rewrite this as shown below. The > >> reason I think this might be faster is that you do not create a > >> temporary samples array that just gets copied. Whether this is faster > >> or not in reality is of course a matter of benchmarking. > >> > >> %% Constants > >> %% Input scale factor > >> -define (S, (1 bsl 23) - 1). > >> %% Block size for DSP (interleaved I and Q) > >> -define (BlockSz, 1024). > >> %% Samples in a Metis frame (interleaved I and Q) > >> -define (MetisSmpls, 63). > >> %% Bytes in a float > >> -define (FloatSz, 4). > >> %% Size of binary for given block size > >> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). > >> > >> do_decode_frame(Frame, State) -> > >> %% Separate the command bytes from the frame data > >> <> = Frame, > >> decode_samples(Rest, State). > >> > >> decode_samples(Frame, State) when size(State) >= ?BinSz -> > >> dsp:process_block(State), > >> decode_samples(Frame, <<>>); > >> decode_samples(< >> Rest/binary>>, State) -> > >> decode_samples(<>; > >> decode_samples(<<>>, State) -> > >> {ok, State}. > >> > >> 2011/3/17 Bob Cowdery > > >>> Hi, > >>> > >>> Am I correct in thinking that the code fragments below will not > copy any > >>> binary data. I've not tested the function yet so it may have bugs but > >>> I'm interested in whether it will be optimal in efficiency. State > starts > >>> off as an empty binary. Frames are decoded and placed in a block. > There > >>> are not an integral number of frames to a block. The function > >>> dsp:process_block() is in another gen-server. > >>> > >>> Thanks > >>> Bob > >>> > >>> %% Constants > >>> %% Input scale factor > >>> -define (S, (1 bsl 23) - 1). > >>> %% Block size for DSP (interleaved I and Q) > >>> -define (BlockSz, 1024). > >>> %% Samples in a Metis frame (interleaved I and Q) > >>> -define (MetisSmpls, 63). > >>> %% Bytes in a float > >>> -define (FloatSz, 4). > >>> %% Size of binary for given block size > >>> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). > >>> > >>> do_decode_frame(Frame, State) -> > >>> %% Separate the command bytes from the frame data > >>> <> = Frame, > >>> %% What's left now is <> > >>> %% This one liner converts the I and Q to scaled floats > >>> Samples = << <<(I/?S):32/float, (Q/?S):32/float>> || > >>> <> <= Rest >>, > >>> CurrentSz = size(State), > >>> FrameSz = size(Samples), > >>> if > >>> (CurrentSz + FrameSz) < ?BinSz -> > >>> NewState = <>; > >>> true -> > >>> %% Binary part of this transfer > >>> CutAt = ?BinSz - CurrentSz - 1, > >>> Bp = binary_part(Samples, {0, CutAt}), > >>> %% Binary part of remainder > >>> Br = binary_part(Samples, {CutAt, FrameSz - CutAt}), > >>> %% Process this block > >>> dsp:process_block(<>), > >>> NewState = Br > >>> end, > >>> {ok, NewState}. > >>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > >>> > > > > > From dsc@REDACTED Fri Mar 18 16:02:58 2011 From: dsc@REDACTED (Dave Challis) Date: Fri, 18 Mar 2011 15:02:58 +0000 Subject: mnesia bags vs sets containing lists References: <4D837422.8080609@ecs.soton.ac.uk> Message-ID: Assuming I wanted to store some simple data structures in mnesia, of the form: {Key, [A, B, C, D, E, ...]}. in which I don't care about the ordering of the list. What would be the advantages/disadvantages of storing it using: a) a table of type set, storing/retrieving the whole list at once, and operating on the list to get specific elements b) a table of type bag, storing/retrieving individual values using mnesia I'm just trying to get a feel for the differences in the above two implementations, rather than having any concrete use in mind. Thanks, -- Dave Challis dsc@REDACTED From mevans@REDACTED Fri Mar 18 16:08:10 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 18 Mar 2011 11:08:10 -0400 Subject: [erlang-questions] mnesia bags vs sets containing lists In-Reply-To: References: <4D837422.8080609@ecs.soton.ac.uk> Message-ID: For the most parts you're probably not going to notice much between the two. What I have noticed with tables of type bag is when the bag gets large then the performance will really slow down, especially on inserts. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Dave Challis Sent: Friday, March 18, 2011 11:03 AM To: erlang-questions Subject: [erlang-questions] mnesia bags vs sets containing lists Assuming I wanted to store some simple data structures in mnesia, of the form: {Key, [A, B, C, D, E, ...]}. in which I don't care about the ordering of the list. What would be the advantages/disadvantages of storing it using: a) a table of type set, storing/retrieving the whole list at once, and operating on the list to get specific elements b) a table of type bag, storing/retrieving individual values using mnesia I'm just trying to get a feel for the differences in the above two implementations, rather than having any concrete use in mind. Thanks, -- Dave Challis dsc@REDACTED ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From alain.odea@REDACTED Fri Mar 18 16:14:01 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 18 Mar 2011 12:44:01 -0230 Subject: [erlang-questions] Jinterface: communication between Java and Erlang using system file In-Reply-To: References: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> Message-ID: <292AB2D7-6961-44B3-ADF7-A66AFD680030@gmail.com> Hi Robby: On 2011-03-18, at 7:45, Robert Raschke wrote: > Hi Zab, > > On Thu, Mar 17, 2011 at 9:02 PM, Zabrane Mickael wrote: > >> Hi guys, >> >> Please, have a look to the following simple Java code >> which try to encode a simple Erlang tuple and save it to disk. >> >> -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 >> >> // encode a simple Erlang tuple {ok, "Hello World"} >> OtpErlangObject[] reply = new OtpErlangObject[2]; >> reply[0] = new OtpErlangAtom("ok"); >> reply[1] = new OtpErlangBinary("Hello World!"); >> OtpErlangTuple tuple = new OtpErlangTuple(reply); >> >> // create an OtpOutputStream object from the binary >> OtpErlangBinary binary = new OtpErlangBinary(tuple); >> OtpOutputStream oos = new OtpOutputStream(binary); >> >> oos.writeTo(fos); >> >> // write the Erlang binary term to a file >> File f = new File ("a.txt"); >> FileOutputStream fos = new FileOutputStream(f); >> >> fos.flush(); >> fos.close(); >> >> -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 >> >> When trying to read it from disk, I got an exception: >> $ erl >> 1> {ok, Bin} = file:read_file("a.txt"). >> {ok,<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109, >> 46,101,114,105,99,115,115,111,110,46,111,...>>} >> 2> binary_to_term(Bin). >> ** exception error: bad argument >> in function binary_to_term/1 >> called as >> binary_to_term(<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109,46, >> >> 101,114,105,99,115,115,111,110,46,111,116,...>>) >> >> >> What am I doing wrong? >> Help appreciated ... >> >> Regards, >> Zab >> >> > I haven't done much Java file writing/reading for a long time, so I've > forgotten a lot. > > But is it possible that the file gets written in "text" mode? And the IO > system helpfully translates line ending characters for you? Java's FileOutputStream writes raw bytes. What you write is what you get. You use wrapper classes like PrintWriter to write text. There may be more packaging or initialization required to use OtpOutputStream in a way that binary_to_term can work with the output. > > Since you're writing binary data, it's always a good idea to make sure > you're writing to a binary file. > > If I remember correctly the Erlang file module functions assume binary by > default. Not sure about Java tho'. > > Robby From t.greenwoodgeer@REDACTED Fri Mar 18 16:14:32 2011 From: t.greenwoodgeer@REDACTED (Todd) Date: Fri, 18 Mar 2011 08:14:32 -0700 Subject: More Erlang Newbie Tutorials Message-ID: <4D8376D8.8060200@gmail.com> I've been continuing to work on entry level tutorials as I learn Erlang. They are posted here: https://github.com/ToddG/experimental/tree/master/erlang/wilderness * Episode-00 : Configure a VMWare guest vm * Episode-02 : Create a basic Erlang App * Episode-03 : Create a 'Time Server' App * Episode-04 : Create a Webmachine Demo App (Webmachine Part I) * Episode-05 : Create a Webmachine Rest Endpoint (Webmachine Part 2) Comments are warmly encouraged. Also, could someone point me at a good example of the proper use of try/catch? I'm approaching Erlang from a Java background, and I'd like to understand the proper way to use exceptions and errors here. -Todd From psa@REDACTED Fri Mar 18 16:30:19 2011 From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=) Date: Fri, 18 Mar 2011 15:30:19 +0000 Subject: [erlang-questions] mnesia bags vs sets containing lists In-Reply-To: References: <4D837422.8080609@ecs.soton.ac.uk> Message-ID: <4D837A8B.3000308@di.uminho.pt> On 3/18/11 3:08 PM, Evans, Matthew wrote: > For the most parts you're probably not going to notice much between the two. > > What I have noticed with tables of type bag is when the bag gets large then the performance will really slow down, especially on inserts. Yes. I don't remember the details, but some years ago when testing bags I was unpleasantly surprised. I summarised it for myself as "bags are evil". :-) Maybe the best way would be to use an ordered set with composite keys of the form {key, A}, {key, B}, etc, mapping to e.g. true. This way you can insert or delete individual objects efficiently or lookup all the objects for a given "key" using select. Regards, Paulo > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Dave Challis > Sent: Friday, March 18, 2011 11:03 AM > To: erlang-questions > Subject: [erlang-questions] mnesia bags vs sets containing lists > > Assuming I wanted to store some simple data structures in mnesia, of the > form: > > {Key, [A, B, C, D, E, ...]}. > > in which I don't care about the ordering of the list. > > What would be the advantages/disadvantages of storing it using: > > a) a table of type set, storing/retrieving the whole list at once, and > operating on the list to get specific elements > > b) a table of type bag, storing/retrieving individual values using mnesia > > I'm just trying to get a feel for the differences in the above two > implementations, rather than having any concrete use in mind. > > Thanks, > From olivier.boudeville@REDACTED Fri Mar 18 18:32:07 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Fri, 18 Mar 2011 18:32:07 +0100 Subject: code_server:call/2 problem? Message-ID: Hi, We are running a distributed Erlang program on a user node from which a number of computing nodes are spawned, via SSH for the remote hosts. To perform the automatic deployment, two deployment-related modules are sent to each of the spawned nodes, using the traditional approach (first a call to code:get_object_code/1 then a rpc:multicall of code:load_binary). However, sometimes (not frequently), with the exact same settings, the first module cannot be deployed successfully. We have indeed: {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ ModuleName, ModuleFilename, ModuleBinary ], Timeout ), that returns: ResList = [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] BadNodes = [] This happens with R14B02, but most probably with previous versions as well. Apparently this happens often (always?) on a node created on the user host. I am pretty sure the deployed node is "fresh" (blank, vanilla). And ignoring the badrpc will result in a undef error as soon as the first function of the first helper module is called, even if delaying the call (a race condition was suspected if ever the actual loading was asynchronous). Would anyone see a cause for such a badarg non-systematic error? Thanks in advance for any hint, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From mevans@REDACTED Fri Mar 18 19:16:50 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 18 Mar 2011 14:16:50 -0400 Subject: [erlang-questions] code_server:call/2 problem? In-Reply-To: References: Message-ID: Do you see any crashes on the remote nodes? It does look like the remote code_server application has got a request, but it for some reason fails (corrupted data perhaps?). I'm wondering if you could attach a remote shell to one of those nodes and trace the code_server module? -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Olivier BOUDEVILLE Sent: Friday, March 18, 2011 1:32 PM To: erlang-questions@REDACTED Subject: [erlang-questions] code_server:call/2 problem? Hi, We are running a distributed Erlang program on a user node from which a number of computing nodes are spawned, via SSH for the remote hosts. To perform the automatic deployment, two deployment-related modules are sent to each of the spawned nodes, using the traditional approach (first a call to code:get_object_code/1 then a rpc:multicall of code:load_binary). However, sometimes (not frequently), with the exact same settings, the first module cannot be deployed successfully. We have indeed: {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ ModuleName, ModuleFilename, ModuleBinary ], Timeout ), that returns: ResList = [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] BadNodes = [] This happens with R14B02, but most probably with previous versions as well. Apparently this happens often (always?) on a node created on the user host. I am pretty sure the deployed node is "fresh" (blank, vanilla). And ignoring the badrpc will result in a undef error as soon as the first function of the first helper module is called, even if delaying the call (a race condition was suspected if ever the actual loading was asynchronous). Would anyone see a cause for such a badarg non-systematic error? Thanks in advance for any hint, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From nico.meyer@REDACTED Fri Mar 18 19:50:45 2011 From: nico.meyer@REDACTED (Nico Meyer) Date: Fri, 18 Mar 2011 19:50:45 +0100 Subject: Jinterface: communication between Java and Erlang using system file In-Reply-To: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> References: <89EDBCB3-01CA-490A-8C4F-7B65670440D7@gmail.com> Message-ID: <1300474245.8327.14.camel@erdnuss1> Hi Zabrane, the external erlang term format always starts with a decimal 131 byte ( http://www.erlang.org/doc/apps/erts/erl_ext_dist.html ). Seems like that is missing from the file. try to add: fos.write(131); somewhere before oos.writeTo(fos); Cheers, Nico Am Donnerstag, den 17.03.2011, 22:02 +0100 schrieb Zabrane Mickael: > Hi guys, > > Please, have a look to the following simple Java code > which try to encode a simple Erlang tuple and save it to disk. > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > // encode a simple Erlang tuple {ok, "Hello World"} > OtpErlangObject[] reply = new OtpErlangObject[2]; > reply[0] = new OtpErlangAtom("ok"); > reply[1] = new OtpErlangBinary("Hello World!"); > OtpErlangTuple tuple = new OtpErlangTuple(reply); > > // create an OtpOutputStream object from the binary > OtpErlangBinary binary = new OtpErlangBinary(tuple); > OtpOutputStream oos = new OtpOutputStream(binary); > > oos.writeTo(fos); > > // write the Erlang binary term to a file > File f = new File ("a.txt"); > FileOutputStream fos = new FileOutputStream(f); > > fos.flush(); > fos.close(); > > -8-8-8-8-8-8-8-8-8-8 snip -8-8-8-8-8-8-8-8-8-8 > > When trying to read it from disk, I got an exception: > $ erl > 1> {ok, Bin} = file:read_file("a.txt"). > {ok,<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109, > 46,101,114,105,99,115,115,111,110,46,111,...>>} > 2> binary_to_term(Bin). > ** exception error: bad argument > in function binary_to_term/1 > called as binary_to_term(<<109,0,0,2,17,172,237,0,5,115,114,0,38,99,111,109,46, > 101,114,105,99,115,115,111,110,46,111,116,...>>) > > > What am I doing wrong? > Help appreciated ... > > Regards, > Zab > > From thatpythonguy@REDACTED Fri Mar 18 20:59:31 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Fri, 18 Mar 2011 15:59:31 -0400 Subject: Erlangers in Montreal? Message-ID: <4D83B9A3.9050801@gmail.com> Hi: Any local Erlang users group in the Montreal area (Canada)? If not, anyone interested in starting one up? Cheers, Ahmed From bsvancara@REDACTED Fri Mar 18 21:18:40 2011 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Fri, 18 Mar 2011 21:18:40 +0100 Subject: Dialyzer crashes under Windows 7 64-bit Message-ID: Dialyzer crashes under Windows 7 64-bit Trying to change a directory in the "Dir" list (left-bottom part of the screen), Dialyzer crashes. Eshell V5.8.3 (abort with ^G) 1> dialyzer:gui(). gs error: user backend died reason {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,idle,1}]} =ERROR REPORT==== 18-Mar-2011::21:12:38 === Bad value on output port 'tcp_inet' =ERROR REPORT==== 18-Mar-2011::21:12:38 === Error in process <0.34.0> with exit value: {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,i dle,1}]} Bob From bob@REDACTED Fri Mar 18 22:33:26 2011 From: bob@REDACTED (Bob Cowdery) Date: Fri, 18 Mar 2011 21:33:26 +0000 Subject: Binary comprehensions again. Message-ID: <4D83CFA6.5030708@bobcowdery.plus.com> Hi This is another part of my code that works well and is pretty efficient (thanks to help from the list). << <<(I/?S):32/float, (Q/?S):32/float>> || <> <= Bin >> I now need to do the same kind of thing in reverse. This is not correct code, it won't even compile but it illustrates the logic I want. << <> || <> <= Bin >>. There are several problems in here and I don't know if the solution can be quite this concise. The input binary is a sequence of two floats which are value +1 to -1. M is a scale factor (2 bsl 15) - 1 which gives a value that fits in a signed 16 bit integer. The output needs to be 4 integers where the last two are just padding for the present. I don't know how to do that so I bound the variables to zero bits. It does not seem to like the function call in there but I need to do the equivalent of a cast somehow. Any hints on how to form that correctly would be appreciated. Bob From pmacgown@REDACTED Sat Mar 19 02:58:25 2011 From: pmacgown@REDACTED (pmacgown@REDACTED) Date: Sat, 19 Mar 2011 01:58:25 +0000 (UTC) Subject: Erlangoliers of Greater Boston Meetup Established Message-ID: <138477725.2814195.1300499905145.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> A small group of Erlang enthusiasts has started up in Boston. Go to Meetup.com and find us. --Peter MacGown - Meetup Organizer From chandrashekhar.mullaparthi@REDACTED Sat Mar 19 07:39:11 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 19 Mar 2011 06:39:11 +0000 Subject: Job posting Message-ID: Who: Everything Everywhere Ltd. (owner of the T-Mobile and Orange GSM networks in the UK) Where: Hatfield, UK, http://bit.ly/fmDALp What: 2 permanent positions in a team which builds core network platforms using mostly Erlang/OTP. We've been using Erlang for over 10 years now, and have 20+ service platforms in production. I'm looking for people with very good Erlang skills. Knowledge of mobile telephony and IN/SS7 is a plus. Competitive salary, company car allowance, bonus scheme. See full job description at http://www.orangejobs.co.uk/fe/tpl_orange01.asp?newms=jj&id=78829&newlang=1 Please apply at above link if you are interested, or email me your CV. cheers Chandru From francesco@REDACTED Sat Mar 19 10:23:34 2011 From: francesco@REDACTED (Francesco Cesarini) Date: Sat, 19 Mar 2011 09:23:34 +0000 Subject: [erlang-questions] Erlangoliers of Greater Boston Meetup Established In-Reply-To: <138477725.2814195.1300499905145.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> References: <138477725.2814195.1300499905145.JavaMail.root@sz0166a.westchester.pa.mail.comcast.net> Message-ID: Hi Peter, Great news. The timing coincides with an Erlang Camp being organised in the area later this year. There is a list of user groups on trapexit where you might want to add yours, making it easier for others to find: http://www.trapexit.org/ErlangUserGroups Francesco pmacgown@REDACTED wrote: A small group of Erlang enthusiasts has started up in Boston. Go to Meetup.com and find us. --Peter MacGown - Meetup Organizer From spawn.think@REDACTED Sat Mar 19 12:12:12 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sat, 19 Mar 2011 12:12:12 +0100 Subject: [erlang-questions] Job posting In-Reply-To: References: Message-ID: It looks interesting, maybe you would like to post it also at Erlang/OTP Professionals group on LinkedIn, career discussions section :) http://www.linkedin.com/groups?careerDiscussion=&gid=90878 On Sat, Mar 19, 2011 at 7:39 AM, Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: > Who: Everything Everywhere Ltd. (owner of the T-Mobile and Orange GSM > networks in the UK) > > Where: Hatfield, UK, http://bit.ly/fmDALp > > What: 2 permanent positions in a team which builds core network platforms > using mostly Erlang/OTP. We've been using Erlang for over 10 years now, and > have 20+ service platforms in production. I'm looking for people with very > good Erlang skills. Knowledge of mobile telephony and IN/SS7 is a plus. > > Competitive salary, company car allowance, bonus scheme. > > See full job description at > http://www.orangejobs.co.uk/fe/tpl_orange01.asp?newms=jj&id=78829&newlang=1 > > Please apply at above link if you are interested, or email me your CV. > > cheers > Chandru > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From fred.hebert@REDACTED Sat Mar 19 17:48:54 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Sat, 19 Mar 2011 12:48:54 -0400 Subject: [erlang-questions] Erlangers in Montreal? In-Reply-To: <4D83B9A3.9050801@gmail.com> References: <4D83B9A3.9050801@gmail.com> Message-ID: <44A65C3D-61B8-4AFA-AC08-41AC3499DE5A@erlang-solutions.com> I'm rather up north in Saguenay if you feel like extending the group a bit ;) I'm not in Montreal very often, but yeah. -- Fred H?bert http://www.erlang-solutions.com On 2011-03-18, at 15:59 PM, Ahmed Al-Saadi wrote: > Hi: > > Any local Erlang users group in the Montreal area (Canada)? If not, anyone interested in starting one up? > > Cheers, > Ahmed > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kostis@REDACTED Sat Mar 19 17:50:41 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 19 Mar 2011 18:50:41 +0200 Subject: [erlang-questions] Dialyzer crashes under Windows 7 64-bit In-Reply-To: References: Message-ID: <4D84DEE1.2080601@cs.ntua.gr> Bohuslav Svancara wrote: > Dialyzer crashes under Windows 7 64-bit > > Trying to change a directory in the "Dir" list (left-bottom part of the > screen), Dialyzer crashes. > > > Eshell V5.8.3 (abort with ^G) > 1> dialyzer:gui(). > gs error: user backend died reason > {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,idle,1}]} > > > =ERROR REPORT==== 18-Mar-2011::21:12:38 === > Bad value on output port 'tcp_inet' > > =ERROR REPORT==== 18-Mar-2011::21:12:38 === > Error in process <0.34.0> with exit value: > {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,i > dle,1}]} I do not have access to such a machine to try this out, but I doubt this is a dialyzer issue. Looks more like a Windows or a gs problem. Can you try dialyzer from the command line and tell us/me what's happening? Kostis From bsvancara@REDACTED Sat Mar 19 18:57:46 2011 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Sat, 19 Mar 2011 18:57:46 +0100 Subject: [erlang-questions] Dialyzer crashes under Windows 7 64-bit In-Reply-To: <4D84DEE1.2080601@cs.ntua.gr> References: <4D84DEE1.2080601@cs.ntua.gr> Message-ID: Hi Kostis! Dialyzer works fine from the command line. It must be something in GUI. Regards, Bob 2011/3/19 Kostis Sagonas > Bohuslav Svancara wrote: > >> Dialyzer crashes under Windows 7 64-bit >> >> Trying to change a directory in the "Dir" list (left-bottom part of the >> screen), Dialyzer crashes. >> >> >> Eshell V5.8.3 (abort with ^G) >> 1> dialyzer:gui(). >> gs error: user backend died reason >> >> {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,idle,1}]} >> >> >> =ERROR REPORT==== 18-Mar-2011::21:12:38 === >> Bad value on output port 'tcp_inet' >> >> =ERROR REPORT==== 18-Mar-2011::21:12:38 === >> Error in process <0.34.0> with exit value: >> >> {{badmatch,{error,einval}},[{gstk_port_handler,output,2},{gstk_port_handler,i >> dle,1}]} >> > > I do not have access to such a machine to try this out, but I doubt this is > a dialyzer issue. Looks more like a Windows or a gs problem. > > Can you try dialyzer from the command line and tell us/me what's happening? > > Kostis > From francis.joanis@REDACTED Sat Mar 19 20:16:07 2011 From: francis.joanis@REDACTED (Francis Joanis) Date: Sat, 19 Mar 2011 15:16:07 -0400 Subject: [erlang-questions] Erlangers in Montreal? In-Reply-To: <44A65C3D-61B8-4AFA-AC08-41AC3499DE5A@erlang-solutions.com> References: <4D83B9A3.9050801@gmail.com> <44A65C3D-61B8-4AFA-AC08-41AC3499DE5A@erlang-solutions.com> Message-ID: <9194F942-2BE1-48B2-823C-E44D377BDBE2@gmail.com> Hi, I know a few guys doing Erlang in Ottawa/Gatineau. We're about 2 hours away. Anybody else from Ottawa by any chance? Cheers, Francis Le 2011-03-19 ? 12:48, Fr?d?ric Trottier-H?bert a ?crit : > I'm rather up north in Saguenay if you feel like extending the group a bit ;) I'm not in Montreal very often, but yeah. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-03-18, at 15:59 PM, Ahmed Al-Saadi wrote: > >> Hi: >> >> Any local Erlang users group in the Montreal area (Canada)? If not, anyone interested in starting one up? >> >> Cheers, >> Ahmed >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ulf.wiger@REDACTED Sat Mar 19 21:32:41 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 19 Mar 2011 21:32:41 +0100 Subject: [erlang-questions] first, next, prev, last in gb_trees In-Reply-To: <37480647E5B94C6E99257BC8027E5437@HALL> References: <641CC0D8-4DFA-48CB-89A3-DDB11F2EB495@erlang-solutions.com> <37480647E5B94C6E99257BC8027E5437@HALL> Message-ID: On 16 Mar 2011, at 20:58, John Hughes wrote: > I'd suggest testing all the properties at once with eqc:module({numtests,1000},gb1). That should work in eqcmini too. Unfortunately, eqcmini (eqc-1.0.1) only has module/1. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From thatpythonguy@REDACTED Sun Mar 20 09:40:17 2011 From: thatpythonguy@REDACTED (Ahmed Al-Saadi) Date: Sun, 20 Mar 2011 04:40:17 -0400 Subject: [erlang-questions] Erlangers in Montreal? In-Reply-To: References: <4D83B9A3.9050801@gmail.com> Message-ID: <4D85BD71.4080300@gmail.com> Hi Kaiduan: I am relatively new to Erlang as I have been doing most of my recent development in Python. Ever since I heard Tim Bray at a software conference discuss the problem of parallelization of computing, I have been searching for a solution. Erlang is *that* solution par excellence! Also, I find the idea of functional programming appealing not only as an enabler of parallel computing, but also as a design tool as it lends itself well to mathematical/logical concepts which appeal to my engineering background. So, yes, I am also passionate about Erlang :) I have been re-writing my company web site using Zotonic, a super-fast Erlang web CMS and, like you, have started an XMPP project that uses ejabberd, though haven't started hacking the source-code. I have created a google group (erlang-montreal@REDACTED) to serve as a hotbed for ideas and a channel through which local meetings can be organized. I tried to create erlang-canada, but that name was taken and I contacted the owner to see if it can be used to start similar groups in different Canadian cities. I invite you, Todd, Fred, Francis and everyone else to join erlang-montreal; if anything, it'll give you an excuse to visit a cool city with vibrant software communities. Cheers, Ahmed On 3/18/11 10:15 PM, Kaiduan Xie wrote: > Ahmed, > > Although I am not in Montreal, I am passionate to Erlang in Waterloo, > Ontario. In my person time, I am writing an Erlang SIP proxy, hacking > Ejabberd. And in work I am trying to push Erlang to my company. > > How about you? Can you share your story with Erlang? > > Best regards, > > /Kaiduan > > On Fri, Mar 18, 2011 at 3:59 PM, Ahmed Al-Saadi wrote: >> Hi: >> >> Any local Erlang users group in the Montreal area (Canada)? If not, anyone >> interested in starting one up? >> >> Cheers, >> Ahmed >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From czinkos@REDACTED Sun Mar 20 12:57:08 2011 From: czinkos@REDACTED (Zsolt Czinkos) Date: Sun, 20 Mar 2011 12:57:08 +0100 Subject: How common using erlang for financial systems? Message-ID: Hello, I'm planning to write my BSc thesis about Erlang (what it is, concurrency, high availability, what it is suitable for and why, etc). I study Information Technology In Economics, so I'd like to demonstrate Erlang capabilities in some finance related area (not telecom where it comes from). So my question is: how common using erlang in financial systems? Can you recommend some articles, URLs, companies ... anything I can start with? Thanks in advance for you help. Best, Zsolt From roberto@REDACTED Sun Mar 20 13:01:52 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2011 13:01:52 +0100 Subject: monitor sockets Message-ID: dear list, is there a way to monitor the closing of a socket? i've been monitoring the closing of sockets with the typical {tcp_closed, } message received, however what if you want to monitor the port from outside the process actually performing the reading of the socket? i've tried link/1 since the man pages state that it could be both a process or a port, however to no avail. any input welcome :) cheers, r. From yrashk@REDACTED Sun Mar 20 14:20:11 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sun, 20 Mar 2011 06:20:11 -0700 (PDT) Subject: [erlang-questions] monitor sockets In-Reply-To: Message-ID: <24123166.304.1300627211633.JavaMail.geo-discussion-forums@preo10> My understanding is that linking to a socket will only work if it's active (or do an explicit recv), so you'll get both tcp_closed *and* an EXIT -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Sun Mar 20 14:20:11 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Sun, 20 Mar 2011 06:20:11 -0700 (PDT) Subject: [erlang-questions] monitor sockets In-Reply-To: Message-ID: <24123166.304.1300627211633.JavaMail.geo-discussion-forums@preo10> My understanding is that linking to a socket will only work if it's active (or do an explicit recv), so you'll get both tcp_closed *and* an EXIT -------------- next part -------------- An HTML attachment was scrubbed... URL: From hokan.stenholm@REDACTED Sun Mar 20 13:49:38 2011 From: hokan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=) Date: Sun, 20 Mar 2011 13:49:38 +0100 Subject: [erlang-questions] How common using erlang for financial systems? In-Reply-To: References: Message-ID: <4D85F7E2.7040400@bredband.net> On 2011-03-20 12.57, Zsolt Czinkos wrote: > Hello, > > I'm planning to write my BSc thesis about Erlang (what it is, > concurrency, high availability, what it is suitable for and why, etc). > I study Information Technology In Economics, so I'd like to > demonstrate Erlang capabilities in some finance related area (not > telecom where it comes from). > > So my question is: how common using erlang in financial systems? Can > you recommend some articles, URLs, companies ... anything I can start > with? > Klarna (formerly Kreditor) a swedish invoicing company, uses a system written in Erlang. Company home page: http://klarna.com - You will probably not get much elang info here. There is at least some info from various programming conferences like the EUC - Erlang User Conference in Stockholm (http://www.erlang.se/euc/index.shtml): Betting on Functional Programming and Winning. - Erik Stenman, Kreditor, Sweden. http://www.erlang.se/euc/06/ Quality Cruising -- Making Java Work for Erlang. - Erik Stenman http://www.erlang.se/euc/07/ Tokyo Cabinet and CouchDB with Mnesia http://www.erlang-factory.com/conference/ErlangUserConference2009/talks Tools@REDACTED http://www.erlang-factory.com/conference/ErlangUserConference2010/talks > > Thanks in advance for you help. > > Best, > > Zsolt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From rapsey@REDACTED Sun Mar 20 15:21:08 2011 From: rapsey@REDACTED (Rapsey) Date: Sun, 20 Mar 2011 15:21:08 +0100 Subject: [erlang-questions] How common using erlang for financial systems? In-Reply-To: References: Message-ID: The stolen Goldman Sachs high frequency trading platform was partly written in Erlang. Sergej On Sun, Mar 20, 2011 at 12:57 PM, Zsolt Czinkos wrote: > Hello, > > I'm planning to write my BSc thesis about Erlang (what it is, > concurrency, high availability, what it is suitable for and why, etc). > I study Information Technology In Economics, so I'd like to > demonstrate Erlang capabilities in some finance related area (not > telecom where it comes from). > > So my question is: how common using erlang in financial systems? Can > you recommend some articles, URLs, companies ... anything I can start > with? > > > > Thanks in advance for you help. > > Best, > > Zsolt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From vinoski@REDACTED Sun Mar 20 15:47:44 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 20 Mar 2011 10:47:44 -0400 Subject: [erlang-questions] monitor sockets In-Reply-To: References: Message-ID: On Sun, Mar 20, 2011 at 8:01 AM, Roberto Ostinelli wrote: > dear list, > > is there a way to monitor the closing of a socket? i've been monitoring the > closing of sockets with the typical {tcp_closed, } message received, > however what if you want to monitor the port from outside the process > actually performing the reading of the socket? > > i've tried link/1 since the man pages state that it could be both a process > or a port, however to no avail. > > any input welcome :) I use this feature and from what I've seen it works well. Can you supply details or code for the kinds of problems you see? In my embedded Yaws system we use TCP offload sockets, and I have a special linked-in driver that interacts with our hardware to open them. When you open an Erlang socket by supplying your own file descriptor, as I do for this TCP offload case, the VM doesn't close the socket file descriptor when its owning process dies as it will for regular sockets. It instead assumes that since you opened the fd separately, you'll take care of closing it separately too. I use socket linking to ensure our underlying offload socket fd also gets closed. --steve From nox@REDACTED Sun Mar 20 17:51:16 2011 From: nox@REDACTED (Anthony Ramine) Date: Sun, 20 Mar 2011 17:51:16 +0100 Subject: [erlang-questions] Binary comprehensions again. In-Reply-To: <4D83CFA6.5030708@bobcowdery.plus.com> References: <4D83CFA6.5030708@bobcowdery.plus.com> Message-ID: Le 18 mars 2011 ? 22:33, Bob Cowdery a ?crit : > Hi > This is another part of my code that works well and is pretty efficient > (thanks to help from the list). > > << <<(I/?S):32/float, (Q/?S):32/float>> || < Q:24/big-signed, _M:16/big-signed>> <= Bin >> > > I now need to do the same kind of thing in reverse. > This is not correct code, it won't even compile but it illustrates the > logic I want. > > << <> || > <> <= Bin >>. > > There are several problems in here and I don't know if the solution can > be quite this concise. The input binary is a sequence of two floats > which are value +1 to -1. M is a scale factor (2 bsl 15) - 1 which gives > a value that fits in a signed 16 bit integer. The output needs to be 4 > integers where the last two are just padding for the present. I don't > know how to do that so I bound the variables to zero bits. It does not > seem to like the function call in there but I need to do the equivalent > of a cast somehow. > > Any hints on how to form that correctly would be appreciated. > > Bob You can't have expressions which aren't constant in patterns so your code obviously does not compile. You want something along those lines: << <<(round(L*?M):24/big-signed, (round(R*?M)):24/big-signed, 0:16>> || <> <= Bin >>. Regards, -- Anthony Ramine Dev:Extend http://dev-extend.eu From roberto@REDACTED Sun Mar 20 18:29:26 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2011 18:29:26 +0100 Subject: [erlang-questions] monitor sockets In-Reply-To: References: Message-ID: 2011/3/20 Steve Vinoski > I use this feature and from what I've seen it works well. Can you > supply details or code for the kinds of problems you see? > > In my embedded Yaws system we use TCP offload sockets, and I have a > special linked-in driver that interacts with our hardware to open > them. When you open an Erlang socket by supplying your own file > descriptor, as I do for this TCP offload case, the VM doesn't close > the socket file descriptor when its owning process dies as it will for > regular sockets. It instead assumes that since you opened the fd > separately, you'll take care of closing it separately too. I use > socket linking to ensure our underlying offload socket fd also gets > closed. > > --steve > hi steve, i'm using the socket in {active, once} mode, and after having read headers and body i proceed to treat a request and i end up in a loop which controls the logic handler. you can see the loop here: https://github.com/ostinelli/misultin/blob/master/src/misultin_http.erl#L297 i tried linking to the port before entering this loop in the call_mfa/2 function, i.e. link(Sock) however the receive... end loop in the socket_loop/3 function never does receive a tcp_closed message. if i insert, as yurii has suggested, at the beginning of the socket_loop/3 function: + misultin_socket:setopts(Sock, [{active, once}], SocketMode) then i do get the {tcp_closed, } message. before adopting this technique, though, i wanted to see which other possibilities i have. thank you, r. From bob@REDACTED Sun Mar 20 19:15:49 2011 From: bob@REDACTED (Bob Cowdery) Date: Sun, 20 Mar 2011 18:15:49 +0000 Subject: [erlang-questions] Binary comprehensions again. In-Reply-To: References: <4D83CFA6.5030708@bobcowdery.plus.com> Message-ID: <4D864455.7030700@bobcowdery.plus.com> Thank you very much both. It does exactly what I want. Hopefully next time I will figure it out myself. Bob On 20/03/2011 13:24, Philip Robinson wrote: > Bob, have you tried this? > > << <<(round(L * ?M)):16/big-signed, (round(R * ?M)):16/big-signed, > 0:16, 0:16>> || > <> <= Bin >>. > > I think that this might be what you are after. On 20/03/2011 16:51, Anthony Ramine wrote: > > You can't have expressions which aren't constant in patterns so your code > obviously does not compile. > > You want something along those lines: > > << <<(round(L*?M):24/big-signed, (round(R*?M)):24/big-signed, 0:16>> || > <> <= Bin >>. > > Regards, > -- > Anthony Ramine > Dev:Extend > http://dev-extend.eu > > > > From goura@REDACTED Mon Mar 21 08:28:22 2011 From: goura@REDACTED (Kazuhiro Ogura) Date: Mon, 21 Mar 2011 16:28:22 +0900 Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem In-Reply-To: <311465162.73841300359543943.JavaMail.root@zimbra> References: <4D81E7BB.8090802@gmail.com> <311465162.73841300359543943.JavaMail.root@zimbra> Message-ID: Hi, I had the same error, and after some struggles, finally I changed the shebang line of lib/erl_docgen/priv/bin/xref_mod_app.escript to the absolute path to escript, and it worked. -#!/usr/bin/env escript +#!/usr/local/bin/escript I don't know why it worked, so I can't tell whether this works for you or not. Sure I have xmlerl-x.y.z in the lib/ dir, and I can load xmerl in a normal erlang shell. Also what's strange is that when I manually execute the command, (which fails complaining about exception about xmerl:export_simple/2 in the make process), it succeeds without any errors. (May be some envvar passed to script is doing something wrong?) Kazuhiro Ogura 2011/3/17 Lukas Larsson : > It seems to me like xmerl is not in the erlang install which is in your path. Check that xmerl-x.y.z is in the lib/ dir of the erlang you have in your path. or alternatively that you can load the xmerl module in a normal erlang shell. > > Lukas > > ----- Original Message ----- > From: "Damian Dobroczy?ski" > To: erlang-questions@REDACTED > Sent: Thursday, 17 March, 2011 11:51:39 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna > Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem > > > Hi. > > I downloaded, compiled and tried to "make docs" and "make install-docs". > The first went fine. The result of the latter is the following: > > escript: exception error: undefined function xmerl:export_simple/2 > ?in function ?erl_eval:do_apply/5 > ?in call from erl_eval:expr/5 > ?in call from erl_eval:exprs/5 > ?in call from erl_eval:local_func/5 > ?in call from escript:interpret/4 > ?in call from escript:start/1 > ?in call from init:start_it/1 > > So, I installed without docs ("make install") and then tried again. The > result was the same. So I removed all escript links and files that I had > had already (some artifacts from previous releases and such) so that I > had only fresh escript in the installation directory and a link in the > prefix (/usr/local/bin) directory. The result was the same. Can somebody > help me, please? > > D. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From boris.muehmer@REDACTED Mon Mar 21 09:41:26 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 21 Mar 2011 09:41:26 +0100 Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem In-Reply-To: References: <4D81E7BB.8090802@gmail.com> <311465162.73841300359543943.JavaMail.root@zimbra> Message-ID: Could You please say, which OS You use? I reported this "strange behavior" for Ubuntu 10.04 / 10.10 some time ago. - boris 2011/3/21 Kazuhiro Ogura : > Hi, > > I had the same error, and after some struggles, > finally I changed the shebang line of > lib/erl_docgen/priv/bin/xref_mod_app.escript > to the absolute path to escript, and it worked. > > -#!/usr/bin/env escript > +#!/usr/local/bin/escript > > I don't know why it worked, > so I can't tell whether this works for you or not. > > Sure I have xmlerl-x.y.z in the lib/ dir, and I can load xmerl in a > normal erlang shell. > Also what's strange is that when I manually execute the command, > (which fails complaining about exception about xmerl:export_simple/2 > in the make process), > it succeeds without any errors. > > (May be some envvar passed to script is doing something wrong?) > > Kazuhiro Ogura > > 2011/3/17 Lukas Larsson : >> It seems to me like xmerl is not in the erlang install which is in your path. Check that xmerl-x.y.z is in the lib/ dir of the erlang you have in your path. or alternatively that you can load the xmerl module in a normal erlang shell. >> >> Lukas >> >> ----- Original Message ----- >> From: "Damian Dobroczy?ski" >> To: erlang-questions@REDACTED >> Sent: Thursday, 17 March, 2011 11:51:39 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem >> >> >> Hi. >> >> I downloaded, compiled and tried to "make docs" and "make install-docs". >> The first went fine. The result of the latter is the following: >> >> escript: exception error: undefined function xmerl:export_simple/2 >> ?in function ?erl_eval:do_apply/5 >> ?in call from erl_eval:expr/5 >> ?in call from erl_eval:exprs/5 >> ?in call from erl_eval:local_func/5 >> ?in call from escript:interpret/4 >> ?in call from escript:start/1 >> ?in call from init:start_it/1 >> >> So, I installed without docs ("make install") and then tried again. The >> result was the same. So I removed all escript links and files that I had >> had already (some artifacts from previous releases and such) so that I >> had only fresh escript in the installation directory and a link in the >> prefix (/usr/local/bin) directory. The result was the same. Can somebody >> help me, please? >> >> D. >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From spawn.think@REDACTED Mon Mar 21 12:16:17 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 21 Mar 2011 12:16:17 +0100 Subject: [erlang-questions] inets httpd problems In-Reply-To: <3717CEF51B134A4F8073627839A9DF4120EBAE9929@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF4120EBAE9929@ESESSCMS0353.eemea.ericsson.se> Message-ID: The quick solution is to add this line to your config BindAddress *|inet If there's an apache like file provided all other configurations provided in start function are ignored except port and address. I'm not sure if this behavior is documented somewhere 2011/3/17 J?zsef B?rces > inets:start(httpd,[{file,ConfFile},{ipfamily,inet}]). > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From olivier.boudeville@REDACTED Mon Mar 21 13:22:37 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Mon, 21 Mar 2011 13:22:37 +0100 Subject: [erlang-questions] code_server:call/2 problem? In-Reply-To: Message-ID: Hi, Thanks for your answer. Indeed, this could have been explanation; however the node is crashing after that error, not before nor "in parallel" to the error. Actually I believe there is a bug in the Erlang runtime. I strongly suspect there is a small time window during which a race condition can occur: apparently code:load_binary can be triggered (thanks to rpc:multicall) on a just-launched node before at least one of its system processes succeeds in registering its name. At least that's what I came to think after having peered at lib/kernel/src/code_server.erl, the badarg that occured may come from the fact that call/2 is called whereas Name is not registered (yet), in: """ call(Name, Req) -> Name ! {code_call, self(), Req}, receive {?MODULE, Reply} -> Reply end. """" As a test, the non-systematic crash which, on our short test case (done on Ubuntu 64-bit running on a 4--core Core i7 laptop), was taking on average 30 seconds (loop of ~15 attempts) before happening, never happened with the same loop being run for more than one hour, once I inserted a timer:sleep(1000) in my deployment manager between the launching of the remote VM and the call to rpc:multicall (knowing that intermediate checkings like Erlang ping of the remote node and checking of the remote Erlang version always succeeded). I suppose there is in the runtime a kind of synchronous barrier where all system processes are checked to be up and ready (including appropriately registered) before serving user-space requests, but probably that at least one system process was forgotten and thus led to such a race condition. Unless I am mistaken? Thanks in advance for any answer, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 mevans@REDACTED Envoy? par : erlang-questions@REDACTED 18/03/2011 19:22 A olivier.boudeville@REDACTED, erlang-questions@REDACTED cc Objet RE: [erlang-questions] code_server:call/2 problem? Do you see any crashes on the remote nodes? It does look like the remote code_server application has got a request, but it for some reason fails (corrupted data perhaps?). I'm wondering if you could attach a remote shell to one of those nodes and trace the code_server module? -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Olivier BOUDEVILLE Sent: Friday, March 18, 2011 1:32 PM To: erlang-questions@REDACTED Subject: [erlang-questions] code_server:call/2 problem? Hi, We are running a distributed Erlang program on a user node from which a number of computing nodes are spawned, via SSH for the remote hosts. To perform the automatic deployment, two deployment-related modules are sent to each of the spawned nodes, using the traditional approach (first a call to code:get_object_code/1 then a rpc:multicall of code:load_binary). However, sometimes (not frequently), with the exact same settings, the first module cannot be deployed successfully. We have indeed: {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ ModuleName, ModuleFilename, ModuleBinary ], Timeout ), that returns: ResList = [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] BadNodes = [] This happens with R14B02, but most probably with previous versions as well. Apparently this happens often (always?) on a node created on the user host. I am pretty sure the deployed node is "fresh" (blank, vanilla). And ignoring the badrpc will result in a undef error as soon as the first function of the first helper module is called, even if delaying the call (a race condition was suspected if ever the actual loading was asynchronous). Would anyone see a cause for such a badarg non-systematic error? Thanks in advance for any hint, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From spawn.think@REDACTED Mon Mar 21 14:43:54 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 21 Mar 2011 14:43:54 +0100 Subject: [erlang-questions] code_server:call/2 problem? In-Reply-To: References: Message-ID: Hi Olivier, I think it has to do with the supervisor children order in kernel app. currently the rpc server is started before the code_server, i'm not sure if there's a good reason for that {ok, {SupFlags, [Rpc, Global, InetDb | DistAC] ++ [NetSup, Glo_grp, File, Code, StdError, User, Config, SafeSupervisor] ++ Timer}} On Mon, Mar 21, 2011 at 1:22 PM, Olivier BOUDEVILLE < olivier.boudeville@REDACTED> wrote: > Hi, > > Thanks for your answer. Indeed, this could have been explanation; however > the node is crashing after that error, not before nor "in parallel" to the > error. > > Actually I believe there is a bug in the Erlang runtime. I strongly > suspect there is a small time window during which a race condition can > occur: apparently code:load_binary can be triggered (thanks to > rpc:multicall) on a just-launched node before at least one of its system > processes succeeds in registering its name. At least that's what I came to > think after having peered at lib/kernel/src/code_server.erl, the badarg > that occured may come from the fact that call/2 is called whereas Name is > not registered (yet), in: > > """ > call(Name, Req) -> > Name ! {code_call, self(), Req}, > receive > {?MODULE, Reply} -> > Reply > end. > """" > > As a test, the non-systematic crash which, on our short test case (done on > Ubuntu 64-bit running on a 4--core Core i7 laptop), was taking on average > 30 seconds (loop of ~15 attempts) before happening, never happened with > the same loop being run for more than one hour, once I inserted a > timer:sleep(1000) in my deployment manager between the launching of the > remote VM and the call to rpc:multicall (knowing that intermediate > checkings like Erlang ping of the remote node and checking of the remote > Erlang version always succeeded). > > I suppose there is in the runtime a kind of synchronous barrier where all > system processes are checked to be up and ready (including appropriately > registered) before serving user-space requests, but probably that at least > one system process was forgotten and thus led to such a race condition. > Unless I am mistaken? > > Thanks in advance for any answer, > Best regards, > > Olivier. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > mevans@REDACTED > Envoy? par : erlang-questions@REDACTED > 18/03/2011 19:22 > > A > olivier.boudeville@REDACTED, erlang-questions@REDACTED > cc > > Objet > RE: [erlang-questions] code_server:call/2 problem? > > > > > > > Do you see any crashes on the remote nodes? > > It does look like the remote code_server application has got a request, > but it for some reason fails (corrupted data perhaps?). I'm wondering if > you could attach a remote shell to one of those nodes and trace the > code_server module? > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Olivier BOUDEVILLE > Sent: Friday, March 18, 2011 1:32 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] code_server:call/2 problem? > > Hi, > > We are running a distributed Erlang program on a user node from which a > number of computing nodes are spawned, via SSH for the remote hosts. To > perform the automatic deployment, two deployment-related modules are sent > to each of the spawned nodes, using the traditional approach (first a call > > to code:get_object_code/1 then a rpc:multicall of code:load_binary). > > However, sometimes (not frequently), with the exact same settings, the > first module cannot be deployed successfully. We have indeed: > > {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ > ModuleName, ModuleFilename, ModuleBinary ], Timeout ), > > that returns: > ResList = > > [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] > BadNodes = [] > > This happens with R14B02, but most probably with previous versions as > well. > Apparently this happens often (always?) on a node created on the user > host. > I am pretty sure the deployed node is "fresh" (blank, vanilla). > And ignoring the badrpc will result in a undef error as soon as the first > function of the first helper module is called, even if delaying the call > (a race condition was suspected if ever the actual loading was > asynchronous). > > Would anyone see a cause for such a badarg non-systematic error? > > Thanks in advance for any hint, > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui > y figurent sont strictement confidentielles. Toute utilisation de ce > Message non conforme ? sa destination, toute diffusion ou toute > publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en > avertir imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete > it and all copies from your system and notify the sender immediately by > return message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui y > figurent sont strictement confidentielles. Toute utilisation de ce Message > non conforme ? sa destination, toute diffusion ou toute publication totale > ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete it > and all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From bob@REDACTED Mon Mar 21 15:09:29 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 21 Mar 2011 14:09:29 +0000 Subject: Float to binary Message-ID: <4D875C19.9050607@bobcowdery.plus.com> If I have a float value. This happens to be a frequency in MHz which I need to convert to a 32 bit integer big-endian format in Hz how do I get the float into a suitable format. The following converts the term into an external term format which is no good for the binary comprehension. How can I convert the value into a binary without getting the external format header or is there a different way to approach this. It seems to know that F is 1.0 as it let me rebind it. 1> F=1.0. 1.0 2> B = term_to_binary(F, [{minor_version, 1}]). <<131,70,63,240,0,0,0,0,0,0>> 3> << <<(round(F*1000000)):32/big>> || <> <= <<63,240,0,0,0,0,0,0>> >>. <<0,15,66,64>> Thanks once again for any help. Bob From alexey.v.romanov@REDACTED Mon Mar 21 15:25:51 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Mon, 21 Mar 2011 17:25:51 +0300 Subject: [erlang-questions] Float to binary In-Reply-To: <4D875C19.9050607@bobcowdery.plus.com> References: <4D875C19.9050607@bobcowdery.plus.com> Message-ID: > It seems to > know that F is 1.0 as it let me rebind it. 1) All variables in list and binary comprehension generators are fresh, so <> <= ... does not rebind F at all. > If I have a float value. This happens to be a frequency in MHz which I > need to convert to a 32 bit integer big-endian format in Hz how do I get > the float into a suitable format. Why do you want to convert it to binary at all, instead of simply `round(F*1000000)`? But if you do (e.g. you want to pack multiple floats there), 1> <<1.0:64/float>>. <<63,240,0,0,0,0,0,0>> Yours, Alexey Romanov On Mon, Mar 21, 2011 at 5:09 PM, Bob Cowdery wrote: > If I have a float value. This happens to be a frequency in MHz which I > need to convert to a 32 bit integer big-endian format in Hz how do I get > the float into a suitable format. ?The following converts the term into > an external term format which is no good for the binary comprehension. > How can I convert the value into a binary without getting the external > 1> F=1.0. > 1.0 > 2> B = term_to_binary(F, [{minor_version, 1}]). > <<131,70,63,240,0,0,0,0,0,0>> > 3> << <<(round(F*1000000)):32/big>> || <> <= > <<63,240,0,0,0,0,0,0>> >>. > <<0,15,66,64>> > > Thanks once again for any help. > > Bob > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From czinkos@REDACTED Mon Mar 21 15:36:12 2011 From: czinkos@REDACTED (Zsolt Czinkos) Date: Mon, 21 Mar 2011 15:36:12 +0100 Subject: [erlang-questions] How common using erlang for financial systems? In-Reply-To: References: Message-ID: Thank you, folks, I'll have at these sites. Zsolt On Sun, Mar 20, 2011 at 3:21 PM, Rapsey wrote: > The stolen Goldman Sachs high frequency trading platform was partly written > in Erlang. > > > Sergej > > > On Sun, Mar 20, 2011 at 12:57 PM, Zsolt Czinkos wrote: >> >> Hello, >> >> I'm planning to write my BSc thesis about Erlang (what it is, >> concurrency, high availability, what it is suitable for and why, etc). >> I study Information Technology In Economics, so I'd like to >> demonstrate Erlang capabilities in some finance related area (not >> telecom where it comes from). >> >> So my question is: how common using erlang in financial systems? Can >> you recommend some articles, URLs, companies ... anything ?I can start >> with? >> >> >> >> Thanks in advance for you help. >> >> Best, >> >> Zsolt >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From bob@REDACTED Mon Mar 21 15:58:54 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 21 Mar 2011 14:58:54 +0000 Subject: [erlang-questions] Float to binary In-Reply-To: References: <4D875C19.9050607@bobcowdery.plus.com> Message-ID: <4D8767AE.5010905@bobcowdery.plus.com> Ah, so I can do: 1> F=7.1. 7.1 2> B = <<(round(F*1000000)):32/big>>. <<0,108,86,96>> This is part of a much larger binary that is sent to a device. Thanks Bob On 21/03/2011 14:25, Alexey Romanov wrote: >> It seems to >> know that F is 1.0 as it let me rebind it. > 1) All variables in list and binary comprehension generators are fresh, so > <> <= ... does not rebind F at all. > >> If I have a float value. This happens to be a frequency in MHz which I >> need to convert to a 32 bit integer big-endian format in Hz how do I get >> the float into a suitable format. > Why do you want to convert it to binary at all, instead of simply > `round(F*1000000)`? > > But if you do (e.g. you want to pack multiple floats there), > > 1> <<1.0:64/float>>. > <<63,240,0,0,0,0,0,0>> > > > Yours, Alexey Romanov > > > > On Mon, Mar 21, 2011 at 5:09 PM, Bob Cowdery wrote: >> If I have a float value. This happens to be a frequency in MHz which I >> need to convert to a 32 bit integer big-endian format in Hz how do I get >> the float into a suitable format. The following converts the term into >> an external term format which is no good for the binary comprehension. >> How can I convert the value into a binary without getting the external >> 1> F=1.0. >> 1.0 >> 2> B = term_to_binary(F, [{minor_version, 1}]). >> <<131,70,63,240,0,0,0,0,0,0>> >> 3> << <<(round(F*1000000)):32/big>> || <> <= >> <<63,240,0,0,0,0,0,0>> >>. >> <<0,15,66,64>> >> >> Thanks once again for any help. >> >> Bob >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From hd2010@REDACTED Mon Mar 21 17:51:18 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 21 Mar 2011 17:51:18 +0100 Subject: Erlang MySQL driver history Message-ID: <4D878206.5030706@eonblast.com> Hi list, I posted a 'history of MySQL drivers' here: http://www.eonblast.com/blog/erlang-mysql-driver-history/ "Open Source Erlang MySQL driver efforts are a fractured matter. You may find yourself digging in the sources to find out about their relationships with each other -- and which one to pick. Here is a brief history." Best, Henning From moxford@REDACTED Mon Mar 21 17:55:29 2011 From: moxford@REDACTED (Mike Oxford) Date: Mon, 21 Mar 2011 09:55:29 -0700 Subject: [erlang-questions] How common using erlang for financial systems? In-Reply-To: References: Message-ID: You might check out the "Big Data, Low Latency" group on LinkedIn. We've had a number of discussions surrounding functional programming as it relates to HF trading. A number of people on there are actually doing it, though some are F#, Haskell, etc. If nothing else, it's people in the realm who're actually doing it and who might be able to give you some good heads-up on it. I think the general outcome from the latest round is that for most of the the HF world, raw speed is king, but if you're not co-located or doing arbs and the like then you can use just about anything else. Erlang is substantially slower than C/C++ (some show 12-16x slower depending on the job.) However, build a system in Erlang and while not a speed demon, it's a hell of a lot easier than scaling out C++ servers with MPI and the like. -mox On Mon, Mar 21, 2011 at 7:36 AM, Zsolt Czinkos wrote: > Thank you, folks, I'll have at these sites. > > Zsolt > > On Sun, Mar 20, 2011 at 3:21 PM, Rapsey wrote: > > The stolen Goldman Sachs high frequency trading platform was partly > written > > in Erlang. > > > > > > Sergej > > > > > > On Sun, Mar 20, 2011 at 12:57 PM, Zsolt Czinkos > wrote: > >> > >> Hello, > >> > >> I'm planning to write my BSc thesis about Erlang (what it is, > >> concurrency, high availability, what it is suitable for and why, etc). > >> I study Information Technology In Economics, so I'd like to > >> demonstrate Erlang capabilities in some finance related area (not > >> telecom where it comes from). > >> > >> So my question is: how common using erlang in financial systems? Can > >> you recommend some articles, URLs, companies ... anything I can start > >> with? > >> > >> > >> > >> Thanks in advance for you help. > >> > >> Best, > >> > >> Zsolt > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kvs@REDACTED Mon Mar 21 18:13:03 2011 From: kvs@REDACTED (Konstantin Sorokin) Date: Mon, 21 Mar 2011 20:13:03 +0300 Subject: [erlang-questions] Erlang MySQL driver history In-Reply-To: <4D878206.5030706@eonblast.com> References: <4D878206.5030706@eonblast.com> Message-ID: So what is the most stable/mature/fast etc. MySQL driver for Erlang at the moment ? Emysql ? On Mon, Mar 21, 2011 at 7:51 PM, Henning Diedrich wrote: > Hi list, > > I posted a 'history of MySQL drivers' here: > http://www.eonblast.com/blog/erlang-mysql-driver-history/ > > "Open Source Erlang MySQL driver efforts are a fractured matter. You may > find yourself digging in the sources to find out about their relationships > with each other -- and which one to pick. Here is a brief history." > > Best, > Henning > > -- Konstantin Sorokin From roberto@REDACTED Mon Mar 21 18:19:21 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 21 Mar 2011 18:19:21 +0100 Subject: [ANNOUNCE] Misultin v0.7-dev Message-ID: dear list, misultin v0.7-dev is currently under review prior being pushed as v0.7. changelog is: - added max_connections options parameter, which specifies maximum concurrent open connections accepted by the server - added post_max_size options parameter, which sets the maximum size of POST data - added get_url_max_size options parameter, which sets the maximum length of URI - added CHUNKED support, both for incoming requests and outgoing responses [thanks to yrashk suggestion] - added trapping of client closing a browser in Comet applications [thanks to yrashk] - added SSL support for websockets [enhancement track #25, thanks to viplifes] - added Comet Long Polling example - added Comet iFrame example - added the killing of alive processes on server shutdown - the GET uri parameters are now also available on POST requests - additional minor adjustments any inputs warmly welcome. https://github.com/ostinelli/misultin cheers, r. From hd2010@REDACTED Mon Mar 21 18:35:10 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Mon, 21 Mar 2011 18:35:10 +0100 Subject: [erlang-questions] Erlang MySQL driver history In-Reply-To: References: <4D878206.5030706@eonblast.com> Message-ID: <4D878C4E.90907@eonblast.com> On 3/21/11 6:13 PM, Konstantin Sorokin wrote: > So what is the most stable/mature/fast etc. MySQL driver for Erlang at > the moment ? Emysql ? Hi Konstantin, Yes. That question was the motivator, obviously and after some months of doubt I guess the history illustrates why. The caveat is: *stable* alone, I would not rule out the ejabberd version as best choice there. It just must have had the biggest actual production exposure. But in a more controlled back end environment probably where certain races may never occur. And it's not like EmySQL has known bugs (issue #7 is interesting but also potentially entirely theoretical in nature). Emysql: https://github.com/Eonblast/Emysql. ejabberd: http://svn.process-one.net/ejabberd-modules/mysql/trunk/ Best, Henning > > On Mon, Mar 21, 2011 at 7:51 PM, Henning Diedrich > wrote: > > Hi list, > > I posted a 'history of MySQL drivers' here: > http://www.eonblast.com/blog/erlang-mysql-driver-history/ > > "Open Source Erlang MySQL driver efforts are a fractured matter. > You may find yourself digging in the sources to find out about > their relationships with each other -- and which one to pick. Here > is a brief history." > > Best, > Henning > > > -- > Konstantin Sorokin From rzezeski@REDACTED Mon Mar 21 19:50:22 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Mon, 21 Mar 2011 14:50:22 -0400 Subject: no next heap size found -2026904952, offset 0 Message-ID: Hi everyone, I've seen this slogan a few times recently in production. Looking at the crash dump I see the error_logger is Garbing (yes, I know about the memory problem with error_logger, I wrote my own handlers but I still must have missed an edge case) and it lists 1359377465 of stack+heap. This is on a 64-bit build of Erlang. I took a peek at erl_gc, copied the impl over to a standalone file, and reproduced the issue but it doesn't make any sense because it's as if my beam binary was built as 32-bit. That is, when I manually set MAX_HEAP_SIZES to 55 (32-bit) I can replicate the error but with 154 (64-bit) it's fine. I can also replicate the error by setting Sint as an int. I checked the source build that was bundled with my release and I see SIZEOF_VOID_P and SIZEOF_LONG set to 8. I also verified HALFWORD_HEAP_EMULATOR is not defined. If there something else I should check? -Ryan From ok@REDACTED Mon Mar 21 22:24:48 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 22 Mar 2011 10:24:48 +1300 Subject: [erlang-questions] Float to binary In-Reply-To: <4D875C19.9050607@bobcowdery.plus.com> References: <4D875C19.9050607@bobcowdery.plus.com> Message-ID: <8CB87164-1340-4085-B20E-EF2B1838035C@cs.otago.ac.nz> On 22/03/2011, at 3:09 AM, Bob Cowdery wrote: > If I have a float value. This happens to be a frequency in MHz which I > need to convert to a 32 bit integer big-endian format in Hz What do you actually mean? What is the range of values? What is their precision? Why won't a simple round(The_Frequency * Scaling_Factor) do the job? From bob@REDACTED Mon Mar 21 23:27:09 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 21 Mar 2011 22:27:09 +0000 Subject: [erlang-questions] Float to binary In-Reply-To: <8CB87164-1340-4085-B20E-EF2B1838035C@cs.otago.ac.nz> References: <4D875C19.9050607@bobcowdery.plus.com> <8CB87164-1340-4085-B20E-EF2B1838035C@cs.otago.ac.nz> Message-ID: <4D87D0BD.3050802@bobcowdery.plus.com> Its part of a binary that gets sent to a device. Its not actually a scaling factor its simply a conversion from MHZ to Hz. 1 MHz = 1000000Hz. The frequency internally is a float but needs to be a 32 bit big-endian number in Hz to send to the device. Bob On 21/03/2011 21:24, Richard O'Keefe wrote: > On 22/03/2011, at 3:09 AM, Bob Cowdery wrote: > >> If I have a float value. This happens to be a frequency in MHz which I >> need to convert to a 32 bit integer big-endian format in Hz > What do you actually mean? > What is the range of values? > What is their precision? > Why won't a simple > > round(The_Frequency * Scaling_Factor) > > do the job? > > From anthonym@REDACTED Tue Mar 22 00:51:28 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 21 Mar 2011 16:51:28 -0700 Subject: Dialyzer bug or some problem with my code Message-ID: <20110321235128.GA16314@alumni.caltech.edu> Hi, Not sure if this is a bug or not, but the attached file exhibits the following. % erlc tmp.erl % erl -eval 'tmp:test(), init:stop()' -noshell All 2 tests passed. % dialyzer -Wno_opaque tmp.erl Checking whether the PLT /Users/molinaro/.dialyzer_plt is up-to-date... yes Proceeding with analysis... tmp.erl:19: Function key_in_dict/2 has no local return Unknown functions: eunit:test/1 done in 0m0.63s done (warnings were emitted) Using R14B02. As far as I can tell the key_in_dict function always returns so I'm not sure why it would have no local return. I can always work around with -Wno_return but I already don't like the fact that I have to work around dict being an opaque type with -Wno_opaque (is that ever going to get fixed?). -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro -------------- next part -------------- -module (tmp). -export ([in_thing/2]). -include_lib("eunit/include/eunit.hrl"). -spec in_thing(_,_) -> boolean(). in_thing (A, B) -> case B of Dict when is_tuple (B) andalso element (1, B) =:= dict -> key_in_dict (A, Dict); List when is_list (B) -> key_in_list (A, List); _ -> false end. -spec key_in_dict(list(),dict()) -> boolean(). key_in_dict (Key, Dict) when is_list (Key) -> dict:is_key (Key, Dict) orelse dict:is_key (list_to_binary(Key), Dict) orelse dict:is_key (list_to_atom(Key), Dict). -spec key_in_list(list(),[{any(),any()}]) -> boolean(). key_in_list (Key, List) when is_list (Key), is_list (List) -> proplists:is_defined (Key, List) orelse proplists:is_defined (list_to_binary(Key), List) orelse proplists:is_defined (list_to_atom(Key), List). -ifdef(EUNIT). -define(ALIST1, [{"A","B"},{"C","D"}]). -define(ALIST2, [{'A',"B"},{'C',"D"}]). -define(ALIST3, [{<<"A">>,"B"},{<<"C">>,"D"}]). in_thing_dict_test () -> ?assertEqual (in_thing("A",dict:from_list (?ALIST1)), true), ?assertEqual (in_thing("A",dict:from_list (?ALIST2)), true), ?assertEqual (in_thing("A",dict:from_list (?ALIST3)), true), ?assertEqual (in_thing("B",dict:from_list (?ALIST1)), false), ?assertEqual (in_thing("B",dict:from_list (?ALIST2)), false), ?assertEqual (in_thing("B",dict:from_list (?ALIST3)), false). in_thing_list_test () -> ?assertEqual (in_thing("A",?ALIST1), true), ?assertEqual (in_thing("A",?ALIST2), true), ?assertEqual (in_thing("A",?ALIST3), true), ?assertEqual (in_thing("B",?ALIST1), false), ?assertEqual (in_thing("B",?ALIST2), false), ?assertEqual (in_thing("B",?ALIST3), false). -endif. From jodie.burch@REDACTED Tue Mar 22 00:56:36 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Mon, 21 Mar 2011 23:56:36 +0000 (GMT) Subject: ErlLounge San Francisco - 24th March 2011 In-Reply-To: <471536524.11181300751612384.JavaMail.root@zimbra> Message-ID: <1518883054.11201300751796190.JavaMail.root@zimbra> Hi all, Don?t forget the ErlLounge in conjunction with the San Francisco Bay Area Erlang Factory this Thursday (March 24th), where we kick off the event with three free talks open to everyone. They start at 5pm at the SFO Hilton, followed by drinks and food at the nearby Coyote Point Yacht Club. A free shuttle bus will be provided. For more information and to register, visit http://www.erlang-factory.com/conference/SFBay2011/erlounge The talks and speakers are: * How Nokia uses Disco to mine terabytes of logs daily - Ville Tuulos * A Tour of Basho's GitHub Repositories: What are they all for? - Scott Lystig Fritchie * Build custom realtime applications with U.C.Engine open source project - Alexandre Eisenchteter Starting at 6pm, there will be a free shuttle bus to take you to the ErlLounge. We have a room set aside for lightning talks. Email us to submit your talk, or if there is space available, sign up on the night. Refreshments, drinks and networking will be plentiful. Just remember to register if you plan on attending, as places are limited. We look forward to seeing you all there! Thanks Jodie From kostis@REDACTED Tue Mar 22 01:26:46 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 22 Mar 2011 02:26:46 +0200 Subject: [erlang-questions] Dialyzer bug or some problem with my code In-Reply-To: <20110321235128.GA16314@alumni.caltech.edu> References: <20110321235128.GA16314@alumni.caltech.edu> Message-ID: <4D87ECC6.2040301@cs.ntua.gr> Anthony Molinaro wrote: > Hi, > > Not sure if this is a bug or not, but the attached file exhibits the > following. > > % erlc tmp.erl > % erl -eval 'tmp:test(), init:stop()' -noshell > All 2 tests passed. > % dialyzer -Wno_opaque tmp.erl > Checking whether the PLT /Users/molinaro/.dialyzer_plt is up-to-date... yes > Proceeding with analysis... > tmp.erl:19: Function key_in_dict/2 has no local return > Unknown functions: > eunit:test/1 > done in 0m0.63s > done (warnings were emitted) > > Using R14B02. As far as I can tell the key_in_dict function always returns > so I'm not sure why it would have no local return. I can always work around > with -Wno_return but I already don't like the fact that I have to work around > dict being an opaque type with -Wno_opaque (is that ever going to get fixed?). You've put yourself in a corner and now you are wondering why you are there... The option -Wno_opaque does not mean what you think it means: it simply shuts off all dialyzer *warnings* about opacity violations in your code. But it does not make dialyzer change its default behavior, which is to consider all opacity violations as failures. Without this option, dialyzer complains that: tmp.erl:11: The call tmp:key_in_dict(A::any(),Dict::tuple()) does not have an opaque term of type dict() as 2nd argument tmp.erl:19: Function key_in_dict/2 has no local return tmp.erl:20: The call dict:is_key(Key::maybe_improper_list(),Dict::tuple()) does not have an opaque term of type dict() as 2nd argument and it's exactly the warning in line 20 which makes dialyzer think that you are doing something wrong there. For better or worse, dict() is defined as an opaque data type, in the Erlang documentation also, which means that you should not really inspect its structure with type tests and element/1 like you do in your code: in_thing (A, B) -> case B of Dict when is_tuple (B) andalso element (1, B) =:= dict -> key_in_dict (A, Dict); List when is_list (B) -> key_in_list (A, List); _ -> false end. In your case you can cheat a bit and "have your cake and eat it too", if you are willing to come to terms that the "let it fail" philosophy is actually a good design principle and your code should not just return false if you happen to pass something unexpected to the second argument of in_thing/2. You can then write your code as: in_thing (A, B) -> case B of List when is_list (B) -> % IMO: this should read is_list(List) here key_in_list (A, List); Dict -> key_in_dict (A, Dict) end. which does NOT inspect the structure of dict(). Dialyzer will then start liking you more. ;) Kostis From ok@REDACTED Tue Mar 22 02:24:14 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 22 Mar 2011 14:24:14 +1300 Subject: [erlang-questions] Float to binary In-Reply-To: <4D87D0BD.3050802@bobcowdery.plus.com> References: <4D875C19.9050607@bobcowdery.plus.com> <8CB87164-1340-4085-B20E-EF2B1838035C@cs.otago.ac.nz> <4D87D0BD.3050802@bobcowdery.plus.com> Message-ID: On 22/03/2011, at 11:27 AM, Bob Cowdery wrote: > Its part of a binary that gets sent to a device. Its not actually a > scaling factor its simply a conversion from MHZ to Hz. 1 MHz = > 1000000Hz. Er, that _is_ a scaling factor. > The frequency internally is a float but needs to be a 32 bit > big-endian number in Hz to send to the device. So will Hz = round(1000000 * MHz) do? From anthonym@REDACTED Tue Mar 22 04:28:50 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 21 Mar 2011 20:28:50 -0700 Subject: [erlang-questions] Dialyzer bug or some problem with my code In-Reply-To: <4D87ECC6.2040301@cs.ntua.gr> References: <20110321235128.GA16314@alumni.caltech.edu> <4D87ECC6.2040301@cs.ntua.gr> Message-ID: <20110322032850.GA16915@alumni.caltech.edu> On Tue, Mar 22, 2011 at 02:26:46AM +0200, Kostis Sagonas wrote: > Anthony Molinaro wrote: > >Hi, > > > > Not sure if this is a bug or not, but the attached file exhibits the > >following. > > > >% erlc tmp.erl > >% erl -eval 'tmp:test(), init:stop()' -noshell > > All 2 tests passed. > >% dialyzer -Wno_opaque tmp.erl > > Checking whether the PLT /Users/molinaro/.dialyzer_plt is up-to-date... yes > > Proceeding with analysis... > >tmp.erl:19: Function key_in_dict/2 has no local return > >Unknown functions: > > eunit:test/1 > > done in 0m0.63s > >done (warnings were emitted) > > > >Using R14B02. As far as I can tell the key_in_dict function always returns > >so I'm not sure why it would have no local return. I can always work around > >with -Wno_return but I already don't like the fact that I have to work around > >dict being an opaque type with -Wno_opaque (is that ever going to get fixed?). > > You've put yourself in a corner and now you are wondering why you > are there... > > The option -Wno_opaque does not mean what you think it means: it > simply shuts off all dialyzer *warnings* about opacity violations in > your code. But it does not make dialyzer change its default > behavior, which is to consider all opacity violations as failures. > > Without this option, dialyzer complains that: > > tmp.erl:11: The call tmp:key_in_dict(A::any(),Dict::tuple()) does > not have an opaque term of type dict() as 2nd argument > tmp.erl:19: Function key_in_dict/2 has no local return > tmp.erl:20: The call > dict:is_key(Key::maybe_improper_list(),Dict::tuple()) does not have > an opaque term of type dict() as 2nd argument > > and it's exactly the warning in line 20 which makes dialyzer think > that you are doing something wrong there. For better or worse, > dict() is defined as an opaque data type, in the Erlang > documentation also, which means that you should not really inspect > its structure with type tests and element/1 like you do in your > code: Okay, that's unfortunate. I realize I'm peeking behind the curtain here but what other choice do I have without any sort of dict:is_dict/1 function. > In your case you can cheat a bit and "have your cake and eat it > too", if you are willing to come to terms that the "let it fail" > philosophy is actually a good design principle and your code should > not just return false if you happen to pass something unexpected to > the second argument of in_thing/2. You can then write your code as: > > in_thing (A, B) -> > case B of > List when is_list (B) -> % IMO: this should read is_list(List) here > key_in_list (A, List); > Dict -> > key_in_dict (A, Dict) > end. The problem here, is this was just a simplified example in the real code I do do something in the third case, so reordering in this way won't quite help me. I guess I can just -Wno_opaque -Wno_return for the moment :( Maybe the OTP team can add an is_dict/1 at some point. Seems like most of the other opaque types have a function to determine if a random term is of that type. The code would be slightly more convoluted as it would be nested conditionals like in_thing (A, B) -> case dict:is_dict (B) of true -> key_in_dict (A, B); false -> case B of List when is_list (List) -> key_in_list (A, B); _ -> false end end. Although not the cleanest at least it should dialyze correctly, and if I wanted to support additional types (for instance maybe I want to allow a set of some sort, I could add it using the is_set/1 function call). Looking through the standard data structures I see that array:is_array/1 gb_sets:is_set/1 ordset:is_set/1 queue:is_queue/1 sets:is_set/1 all exists, but these are missing dict:is_dict/1 gb_trees:is_gb_tree/1 orddict:is_orddict/1 Is there any reason these haven't been added to OTP yet? Something about these last three structures which have kept them from getting a function to determine if a term is of the given type? (they all seem to be a dictionary like structure, so maybe something about dict structures means we can't have an is_dict/1?). Just wondering, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From kenrobinsonster@REDACTED Tue Mar 22 04:47:43 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Tue, 22 Mar 2011 13:47:43 +1000 Subject: Nested finite state machines - Existing Implementations? Message-ID: Hi All, I have a standard which I'm implementing which calls for nested state machines which have push and pop states and entry and exit actions. I'm having difficulty using gen_fsm to do this. I've been looking at plain_fsm by Ulf Wiger which looks promising. Before I reinvent the wheel, is there any existing fully fledged nested state machines out there? -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From emmiller@REDACTED Tue Mar 22 05:25:56 2011 From: emmiller@REDACTED (Evan Miller) Date: Mon, 21 Mar 2011 23:25:56 -0500 Subject: ErlyDTL 0.7.0 is available Message-ID: ErlyDTL compiles Django Template Language to Erlang BEAM code. ErlyDTL 0.7.0, released today, implements 100% of Django filters and several new tags. Download: http://code.google.com/p/erlydtl/downloads/list Project homepage: http://code.google.com/p/erlydtl/ Changes since the last release: * Feature: Boolean operators in "if" tags. * Feature: {% empty %} block in for loops * Feature: translation funs support run-time translation * Feature: "only" and "with" keywords in "include" tag * Feature: ISO-8601 date formatting * Feature: single-argument form of "date" filter * Feature: pass in a helper module to process custom tags * Feature: compile a directory of templates into a helper module * Bugfix: escapejs uses \u syntax * New tags: + filter + spaceless + ssi + templatetag + widthratio + with * New filters (most of these thanks to Drew Gulino) + addslashes + cut + default + dictsort + dictsortreversed + divisibleby + filesizeformat + floatformat + get_digit + iriencode + linebreaks + linenumbers + makelist + phone2numeric + pluralize + random + random_num + random_range + removetags + safe + safeseq + slice + slugify + stringformat + striptags + time + timesince + timeuntil + title + truncatewords_html + unordered_list + urlize + urlizetrunc + wordcount + wordwrap + yesno Questions can be directed to our mailing list: http://groups.google.com/group/erlydtl-users Evan Miller From emmiller@REDACTED Tue Mar 22 05:44:25 2011 From: emmiller@REDACTED (Evan Miller) Date: Mon, 21 Mar 2011 23:44:25 -0500 Subject: Chicago Boss 0.5.0 is available Message-ID: Chicago Boss is a Rails-like web MVC framework for Erlang. Chicago Boss 0.5.0, released today, includes support for several databases, Comet messaging, and more. Download: http://www.chicagoboss.org/projects/chicagoboss/files Project homepage: http://www.chicagoboss.org/ Changes since 0.4.0: New databases * MySQL * PostgreSQL * MongoDB New features * Channel-based message queue for implementing Comet * Sessions (ETS and Mnesia) * Hot code upgrades in production * Support for Windows * Support for SMTP relays * Builds with Rebar Questions can be directed to our mailing list: http://groups.google.com/group/chicagoboss Evan Miller From joe@REDACTED Tue Mar 22 05:51:31 2011 From: joe@REDACTED (Joe Williams) Date: Mon, 21 Mar 2011 21:51:31 -0700 Subject: wx app file parsing error Message-ID: <486130C5-A5DE-4B64-BAEA-D4CD80F847A2@joetify.com> Anyone else run into this issue? 1> file:consult("/usr/local/lib/erlang/lib/wx-0.98.9/ebin/wx.app"). {error,{26,erl_parse, ["syntax error before: ","wxAcceleratorTable"]}} At least on my system (OSX) it looks like the file contains lines like "wxAcceleratorEntry,n wxAcceleratorTable,n wxArtProvider,n ...". On my linux box it seems fine. -Joe Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From ok@REDACTED Tue Mar 22 06:11:58 2011 From: ok@REDACTED (ok@REDACTED) Date: Tue, 22 Mar 2011 18:11:58 +1300 Subject: [erlang-questions] Dialyzer bug or some problem with my code In-Reply-To: <20110322032850.GA16915@alumni.caltech.edu> References: <20110321235128.GA16314@alumni.caltech.edu> <4D87ECC6.2040301@cs.ntua.gr> <20110322032850.GA16915@alumni.caltech.edu> Message-ID: > Okay, that's unfortunate. I realize I'm peeking behind the curtain here > but > what other choice do I have without any sort of dict:is_dict/1 function. The fact that you want to ask the question means that you have some datum X which *might* be a dict or it might be something else. Since dict is not a primitive Erlang type, there is *no* "something else" which will *certainly* not be a possible dict in some future revision. For example, the dict maintainers might decide to put the data in a table of some sort and simply pass out some sort of token, an integer perhaps. Why isn't there a dict:is_dict/1? For the converse reason: there is no possible data structure that the dict module can use that cannot be forged or accidentally infringed on by an outside data type. The fact that Erlang has no unforgeable abstract data types is in some ways regrettable and in some ways useful. The Dialyzer does much to reduce the regrettable aspects. The only way to make the test safely is to put your *own* wrappers around abstract data types. *YOUR* code should be responsible for passing around {dict,X} or {non_dict,Y} terms, wrapping and unwrapping when necessary. This the Dialyzer can help you with. From bob@REDACTED Tue Mar 22 09:12:54 2011 From: bob@REDACTED (Bob Cowdery) Date: Tue, 22 Mar 2011 08:12:54 +0000 Subject: [erlang-questions] Float to binary In-Reply-To: References: <4D875C19.9050607@bobcowdery.plus.com> <8CB87164-1340-4085-B20E-EF2B1838035C@cs.otago.ac.nz> <4D87D0BD.3050802@bobcowdery.plus.com> Message-ID: <4D885A06.1040203@bobcowdery.plus.com> On 22/03/2011 01:24, Richard O'Keefe wrote: > On 22/03/2011, at 11:27 AM, Bob Cowdery wrote: > >> Its part of a binary that gets sent to a device. Its not actually a >> scaling factor its simply a conversion from MHZ to Hz. 1 MHz = >> 1000000Hz. > Er, that _is_ a scaling factor. Yes, you are right in that it is a multiplier. It's just that I differentiate between a scaling factor which changes the value and something that changes only the units, which I call a conversion factor. I don't think its worth arguing over. >> The frequency internally is a float but needs to be a 32 bit >> big-endian number in Hz to send to the device. > So will Hz = round(1000000 * MHz) do? Exactly what I have. Then I need to inset it in a binary as part of the protocol packet to a device. I keep all the protocol fragments as binaries and just combine them in different ways depending on what I am sending to the device. If that's wrong please say how else you think that should be done. I'm still a beginner in Erlang! From ulf.wiger@REDACTED Tue Mar 22 08:50:08 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 22 Mar 2011 08:50:08 +0100 Subject: [erlang-questions] Nested finite state machines - Existing Implementations? In-Reply-To: References: Message-ID: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> Hi Ken, I don't think nested state machines are used much in the erlang world, since it's generally considered detrimental to "complify" the FSM running in a single process. OTOH, sometimes, you need to, in order to implement some behaviour. One of the examples I've used in presentations was when implementing a Diameter stack (rfc3588). It (arbitrarily) combines different FSMs into a single thread, and this caused me to refactor the spec into multiple cooperating FSMs (using plain_fsm). The price you pay is that it becomes hard to check that your code actually matches the spec - even though it became easier to see that the code did what I think was the intention? At AXD301 (a long, long time ago, it seems), we had the need for at least two levels of nesting, for a call-control FSM and a transport FSM - one for each call leg. We did this by making the call-control FSM an active process, a la plain_fsm, and implementing the transport FSMs as gen_fsm-style callbacks (but dispatched from within the CC FSM). We went through many iterations, so it's quite likely that we eventually departed from this model and implemented all FSMs as separate processes. And going for as many processes as the problem called for was one of the driving forces behind gproc, which can serve as a index for locating the FSM you need (based on ordered_set semantics and supporting QLC, you should be able to efficiently select even a subset of multiple FSMs). http://github.com/esl/gproc In my presentation, "Death by Accidental Complexity", I use a parameterised bare-bones event loop for FSM programming, which I also extend with a "select vector" to support a form of selective receive in the FSM callbacks. Most of these things, and more (e.g. a link to ExAT) can be found in the following thread: http://forum.trapexit.org/viewtopic.php?p=15811&sid=197b687b90d851e0b7ffdaa4f1465f8f If memory serves, Joel Reymont solved the problem to his own satisfaction in the poker server, but my quick late-night googling failed to reveal a good link. Joel? BR, Ulf W On 22 Mar 2011, at 04:47, Ken Robinson wrote: > Hi All, > I have a standard which I'm implementing which calls for nested state > machines which have push and pop states and entry and exit actions. > I'm having difficulty using gen_fsm to do this. I've been looking at > plain_fsm by Ulf Wiger which looks promising. Before I reinvent the > wheel, is there any existing fully fledged nested state machines out > there? > > -- > regards, > Ken Robinson > Mob +61438681120 > Home +61738523767 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Tue Mar 22 09:40:36 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 22 Mar 2011 09:40:36 +0100 Subject: code:lib_dir weirdness Message-ID: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> I ran into some problems with paths in an eunit test using the OTP slave library. The main issue was that it didn't work the same in another environment way as on my machine. After realising that my ERL_LIBS setting covered up some sins (duh!), I tried using code:lib_dir() to find the proper app directory (as the code under test was loaded from the rebar-generated .eunit directory). To my surprise, the call to code:lib_dir/1 returned {error, bad_name} on the other machine, but worked on mine. I thought I understood code:lib_dir/1, but after reading the docs, I decided I probably don't. The doc says that lib_dir/1: Returns {error, bad_name} if Name is not the name of an application under $OTPROOT/lib or on a directory referred to via the ERL_LIBS environment variable. (but above it, it mentions regular directories in the code path, which made me wonder if the above is the whole truth.) Finally, I came to the following conclusion: $ pwd /Users/uwiger/ETC/git/jobs $ erl -pa ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> code:lib_dir(jobs). {error,bad_name} 2> code:add_path("/Users/uwiger/ETC/git/jobs/ebin"). true 3> code:lib_dir(jobs). "/Users/uwiger/ETC/git/jobs" Ok, so far, I'm fairly confused. A few possible explanations exist. Next try: $ erl -pa `pwd`/ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> code:lib_dir(jobs). "/Users/uwiger/ETC/git/jobs" 2> os:getenv("ERL_LIBS"). [] Apparently, code:lib_dir/1 will return a good result if a fully qualified path name exists in the code path to the application in question. A relative path won't do. I didn't get that from the documentation, nor do I know if it's the intended behaviour. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From moxford@REDACTED Tue Mar 22 09:59:41 2011 From: moxford@REDACTED (Mike Oxford) Date: Tue, 22 Mar 2011 01:59:41 -0700 Subject: [erlang-questions] code:lib_dir weirdness In-Reply-To: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> References: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> Message-ID: Did you happen to try using "-pa ./ebin" I have no reference, but I'm curious if the leading "./" would help. -mox On Tue, Mar 22, 2011 at 1:40 AM, Ulf Wiger wrote: > > I ran into some problems with paths in an eunit test using the OTP slave > library. > > The main issue was that it didn't work the same in another environment way > as on my machine. > > After realising that my ERL_LIBS setting covered up some sins (duh!), I > tried using > code:lib_dir() to find the proper app directory (as the code under test was > loaded from > the rebar-generated .eunit directory). > > To my surprise, the call to code:lib_dir/1 returned {error, bad_name} on > the other machine, > but worked on mine. I thought I understood code:lib_dir/1, but after > reading the docs, I decided > I probably don't. > > The doc says that lib_dir/1: > > Returns {error, bad_name} if Name is not the name of an application under > $OTPROOT/lib or on a directory referred to via the ERL_LIBS environment > variable. > > (but above it, it mentions regular directories in the code path, which made > me wonder if the above > is the whole truth.) > > Finally, I came to the following conclusion: > > $ pwd > /Users/uwiger/ETC/git/jobs > $ erl -pa ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> code:lib_dir(jobs). > {error,bad_name} > 2> code:add_path("/Users/uwiger/ETC/git/jobs/ebin"). > true > 3> code:lib_dir(jobs). > "/Users/uwiger/ETC/git/jobs" > > Ok, so far, I'm fairly confused. A few possible explanations exist. Next > try: > > $ erl -pa `pwd`/ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > 1> code:lib_dir(jobs). > "/Users/uwiger/ETC/git/jobs" > 2> os:getenv("ERL_LIBS"). > [] > > Apparently, code:lib_dir/1 will return a good result if a fully qualified > path name exists in > the code path to the application in question. A relative path won't do. I > didn't get that > from the documentation, nor do I know if it's the intended behaviour. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From olivier.boudeville@REDACTED Tue Mar 22 10:03:03 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 22 Mar 2011 10:03:03 +0100 Subject: [erlang-questions] code_server:call/2 problem? In-Reply-To: Message-ID: Hi, Thanks for that information. Indeed, shouldn't the kernel ensure that the code_server is not only started but also already up and running before putting the rpc server online? Is there any Erlang maintainer willing to comment on that? Otherwise I guess the user code would have to poll the registering service for the appropriate name (code_server or alike) to be declared before triggering the code-related operations, which would be a bit awkward/unfortunate! I suppose that rpc depends on code_server (and on many other services), but not the other way round? Maybe it would be worth to establish a dependency graph of services to ensure that their starting up is properly synchronised? (might be overkill) Thanks in advance for any hint, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 spawn.think@REDACTED Envoy? par : erlang-questions@REDACTED 21/03/2011 14:46 A olivier.boudeville@REDACTED cc mevans@REDACTED, erlang-questions@REDACTED Objet Re: [erlang-questions] code_server:call/2 problem? Hi Olivier, I think it has to do with the supervisor children order in kernel app. currently the rpc server is started before the code_server, i'm not sure if there's a good reason for that {ok, {SupFlags, [Rpc, Global, InetDb | DistAC] ++ [NetSup, Glo_grp, File, Code, StdError, User, Config, SafeSupervisor] ++ Timer}} On Mon, Mar 21, 2011 at 1:22 PM, Olivier BOUDEVILLE < olivier.boudeville@REDACTED> wrote: > Hi, > > Thanks for your answer. Indeed, this could have been explanation; however > the node is crashing after that error, not before nor "in parallel" to the > error. > > Actually I believe there is a bug in the Erlang runtime. I strongly > suspect there is a small time window during which a race condition can > occur: apparently code:load_binary can be triggered (thanks to > rpc:multicall) on a just-launched node before at least one of its system > processes succeeds in registering its name. At least that's what I came to > think after having peered at lib/kernel/src/code_server.erl, the badarg > that occured may come from the fact that call/2 is called whereas Name is > not registered (yet), in: > > """ > call(Name, Req) -> > Name ! {code_call, self(), Req}, > receive > {?MODULE, Reply} -> > Reply > end. > """" > > As a test, the non-systematic crash which, on our short test case (done on > Ubuntu 64-bit running on a 4--core Core i7 laptop), was taking on average > 30 seconds (loop of ~15 attempts) before happening, never happened with > the same loop being run for more than one hour, once I inserted a > timer:sleep(1000) in my deployment manager between the launching of the > remote VM and the call to rpc:multicall (knowing that intermediate > checkings like Erlang ping of the remote node and checking of the remote > Erlang version always succeeded). > > I suppose there is in the runtime a kind of synchronous barrier where all > system processes are checked to be up and ready (including appropriately > registered) before serving user-space requests, but probably that at least > one system process was forgotten and thus led to such a race condition. > Unless I am mistaken? > > Thanks in advance for any answer, > Best regards, > > Olivier. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > mevans@REDACTED > Envoy? par : erlang-questions@REDACTED > 18/03/2011 19:22 > > A > olivier.boudeville@REDACTED, erlang-questions@REDACTED > cc > > Objet > RE: [erlang-questions] code_server:call/2 problem? > > > > > > > Do you see any crashes on the remote nodes? > > It does look like the remote code_server application has got a request, > but it for some reason fails (corrupted data perhaps?). I'm wondering if > you could attach a remote shell to one of those nodes and trace the > code_server module? > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Olivier BOUDEVILLE > Sent: Friday, March 18, 2011 1:32 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] code_server:call/2 problem? > > Hi, > > We are running a distributed Erlang program on a user node from which a > number of computing nodes are spawned, via SSH for the remote hosts. To > perform the automatic deployment, two deployment-related modules are sent > to each of the spawned nodes, using the traditional approach (first a call > > to code:get_object_code/1 then a rpc:multicall of code:load_binary). > > However, sometimes (not frequently), with the exact same settings, the > first module cannot be deployed successfully. We have indeed: > > {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ > ModuleName, ModuleFilename, ModuleBinary ], Timeout ), > > that returns: > ResList = > > [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] > BadNodes = [] > > This happens with R14B02, but most probably with previous versions as > well. > Apparently this happens often (always?) on a node created on the user > host. > I am pretty sure the deployed node is "fresh" (blank, vanilla). > And ignoring the badrpc will result in a undef error as soon as the first > function of the first helper module is called, even if delaying the call > (a race condition was suspected if ever the actual loading was > asynchronous). > > Would anyone see a cause for such a badarg non-systematic error? > > Thanks in advance for any hint, > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui > y figurent sont strictement confidentielles. Toute utilisation de ce > Message non conforme ? sa destination, toute diffusion ou toute > publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en > avertir imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete > it and all copies from your system and notify the sender immediately by > return message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui y > figurent sont strictement confidentielles. Toute utilisation de ce Message > non conforme ? sa destination, toute diffusion ou toute publication totale > ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete it > and all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From mazen.harake@REDACTED Tue Mar 22 10:09:37 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 22 Mar 2011 10:09:37 +0100 Subject: [erlang-questions] code:lib_dir weirdness In-Reply-To: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> References: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> Message-ID: This case clause ( https://github.com/erlang/otp/blob/dev/lib/kernel/src/code_server.erl#L775 ) seem to check for "ebin" only and then assumes that the directory before it is an application (#778) or an application with the "app-vsn" format (#790) for it to return. Didn't dig into it too much but it should work doing "erl -pa ../app/ebin" /M On 22 March 2011 09:40, Ulf Wiger wrote: > > I ran into some problems with paths in an eunit test using the OTP slave library. > > The main issue was that it didn't work the same in another environment way as on my machine. > > After realising that my ERL_LIBS setting covered up some sins (duh!), I tried using > code:lib_dir() to find the proper app directory (as the code under test was loaded from > the rebar-generated .eunit directory). > > To my surprise, the call to code:lib_dir/1 returned {error, bad_name} on the other machine, > but worked on mine. I thought I understood code:lib_dir/1, but after reading the docs, I decided > I probably don't. > > The doc says that lib_dir/1: > > Returns {error, bad_name} if Name is not the name of an application under $OTPROOT/lib or on a directory referred to via the ERL_LIBS environment variable. > > (but above it, it mentions regular directories in the code path, which made me wonder if the above > is the whole truth.) > > Finally, I came to the following conclusion: > > $ pwd > /Users/uwiger/ETC/git/jobs > $ erl -pa ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 ?(abort with ^G) > 1> code:lib_dir(jobs). > {error,bad_name} > 2> code:add_path("/Users/uwiger/ETC/git/jobs/ebin"). > true > 3> code:lib_dir(jobs). > "/Users/uwiger/ETC/git/jobs" > > Ok, so far, I'm fairly confused. A few possible explanations exist. Next try: > > $ erl -pa `pwd`/ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 ?(abort with ^G) > 1> code:lib_dir(jobs). > "/Users/uwiger/ETC/git/jobs" > 2> os:getenv("ERL_LIBS"). > [] > > Apparently, code:lib_dir/1 will return a good result if a fully qualified path name exists in > the code path to the application in question. A relative path won't do. I didn't get that > from the documentation, nor do I know if it's the intended behaviour. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From ulf.wiger@REDACTED Tue Mar 22 10:13:40 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 22 Mar 2011 10:13:40 +0100 Subject: [erlang-questions] code:lib_dir weirdness In-Reply-To: References: <700CF19D-853C-4451-A390-55579CAEFB9A@erlang-solutions.com> Message-ID: <7599479E-F462-473F-B535-B7AB1E864EF9@erlang-solutions.com> On 22 Mar 2011, at 10:09, Mazen Harake wrote: > Didn't dig into it too much but it should work doing "erl -pa ../app/ebin" It does indeed. BR, Ulf Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From caox@REDACTED Tue Mar 22 10:16:29 2011 From: caox@REDACTED (caox) Date: Tue, 22 Mar 2011 17:16:29 +0800 Subject: how to create compile warning for deprecated functions? Message-ID: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> Hi, I have some lib functions will be deprecated in next release of our repo. How could I add a warning during compiling for other developers just as the module regexp does: src/lib/email/add_phone.erl:23: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead regards, caox From jean-sebastien.pedron@REDACTED Tue Mar 22 10:22:35 2011 From: jean-sebastien.pedron@REDACTED (=?ISO-8859-1?Q?Jean-S=E9bastien_P=E9dron?=) Date: Tue, 22 Mar 2011 10:22:35 +0100 Subject: Release packaging (on Debian) and live upgrade Message-ID: <4D886A5B.1090405@dumbbell.fr> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi! I just wanted to let you know that I written an article about how we package our Erlang releases at work[1]. Yakaz.com is heavily based on Erlang and runs on Debian servers. We wanted to package our Erlang releases as Debian packages but still support live upgrade of a running service. Here's the link: http://engineering.yakaz.com/erlang-packaging-on-debian-and-live-upgrade.html What about you? How do you deploy Erlang releases/applications? [1] Yakaz, free classified ads, http://www.yakaz.com/ - -- Jean-S?bastien P?dron Yakaz (http://www.yakaz.com/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2IalsACgkQa+xGJsFYOlO5TgCfeZSFwNKCNLP8GuZVjWbwhfzd PewAoLTN+z2oU3ZV/8m0ZOUZQ9r9CEbB =4Q/w -----END PGP SIGNATURE----- From ingela@REDACTED Tue Mar 22 11:03:57 2011 From: ingela@REDACTED (Ingela Andin) Date: Tue, 22 Mar 2011 11:03:57 +0100 Subject: [erlang-questions] inets httpd problems In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120EBAE9929@ESESSCMS0353.eemea.ericsson.se> Message-ID: Hi! If you specify the file or prolist_file option all other options should be in the file. >From the inets httpd-docs: "When the web server is started at application start time the properties should be fetched from a configuration file that could consist of a regular erlang property list, e.i. [{Option, Value}] where Option = property() and Value = term(), followed by a full stop, or for backwards compatibility an Apache like configuration file. If the web server is started dynamically at runtime you may still specify a file but you could also just specify the complete property list. {proplist_file, path()} If this property is defined inets will expect to find all other properties defined in this file. Note that the file must include all properties listed under mandatory properties. {file, path()} If this property is defined inets will expect to find all other properties defined in this file, that uses Apache like syntax. " Note that the properties proplist_file and file are mutually exclusive. Regards Ingela Erlang OTP team - Ericsson AB 2011/3/21 Ahmed Omar : > The quick solution is to add this line to your config > BindAddress *|inet > > If there's an apache like file provided all other configurations provided in > start function are ignored except port and address. I'm not sure if this > behavior is documented somewhere > > 2011/3/17 J?zsef B?rces > >> inets:start(httpd,[{file,ConfFile},{ipfamily,inet}]). >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From nick@REDACTED Tue Mar 22 11:08:54 2011 From: nick@REDACTED (Niclas Eklund) Date: Tue, 22 Mar 2011 11:08:54 +0100 Subject: [erlang-questions] how to create compile warning for deprecated functions? In-Reply-To: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> References: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> Message-ID: Hi! Checkout Xref - http://www.erlang.org/doc/man/xref.html Best Regards, Niclas @ Erlang/OTP On Tue, 22 Mar 2011, caox wrote: > Hi, > > I have some lib functions will be deprecated in next release of our repo. How could I add a warning during compiling for other developers just as the module regexp does: > > src/lib/email/add_phone.erl:23: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead > > regards, > caox > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jozsef.berces@REDACTED Tue Mar 22 11:14:15 2011 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Tue, 22 Mar 2011 11:14:15 +0100 Subject: [erlang-questions] inets httpd problems In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF4120EBAE9929@ESESSCMS0353.eemea.ericsson.se> Message-ID: <3717CEF51B134A4F8073627839A9DF4120EBAE9DE2@ESESSCMS0353.eemea.ericsson.se> Hi, Thanks for the answer. Unfortunately, it does not matter if the configuration is coming from the file or not, the default fallback option does not seem to work properly (ESOE Vista). It accepts only IPv6. If the browser sends IPv4 it just does not answer. After running the trace suggested by bmk@REDACTED (inets:enable_trace(max, io, httpd)), it seems that the problem is at lower level as the trace did not produce any printout related to the HTTP queries (it had a lot of other printouts, for example, during the start up of httpd). Anyway, for me setting BindAddress 127.0.0.1|inet solved the problem as I need local access only. But if I wanted httpd to listen at all of the IP addresses of the computer (i.e. no BindAddress) it would not work because in that case it always wants IPv6 and (as the trace showed) the HTTP request does not even come to httpd, it dies at a lower layer. BR, Jozsef -----Original Message----- From: ingela.andin@REDACTED [mailto:ingela.andin@REDACTED] On Behalf Of Ingela Andin Sent: Tuesday, March 22, 2011 17:04 To: Ahmed Omar Cc: J?zsef B?rces; erlang-questions@REDACTED Subject: Re: [erlang-questions] inets httpd problems Hi! If you specify the file or prolist_file option all other options should be in the file. >From the inets httpd-docs: "When the web server is started at application start time the properties should be fetched from a configuration file that could consist of a regular erlang property list, e.i. [{Option, Value}] where Option = property() and Value = term(), followed by a full stop, or for backwards compatibility an Apache like configuration file. If the web server is started dynamically at runtime you may still specify a file but you could also just specify the complete property list. {proplist_file, path()} If this property is defined inets will expect to find all other properties defined in this file. Note that the file must include all properties listed under mandatory properties. {file, path()} If this property is defined inets will expect to find all other properties defined in this file, that uses Apache like syntax. " Note that the properties proplist_file and file are mutually exclusive. Regards Ingela Erlang OTP team - Ericsson AB 2011/3/21 Ahmed Omar : > The quick solution is to add this line to your config BindAddress > *|inet > > If there's an apache like file provided all other configurations > provided in start function are ignored except port and address. I'm > not sure if this behavior is documented somewhere > > 2011/3/17 J?zsef B?rces > >> inets:start(httpd,[{file,ConfFile},{ipfamily,inet}]). >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From spawn.think@REDACTED Tue Mar 22 11:38:22 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 22 Mar 2011 11:38:22 +0100 Subject: [ANN] Erlang/OTP professionals group on LinkedIn surpassed 1000 Members! Message-ID: http://lnkd.in/hwXMqt -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From freza@REDACTED Tue Mar 22 12:37:54 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 22 Mar 2011 11:37:54 +0000 Subject: [erlang-questions] wx app file parsing error In-Reply-To: <486130C5-A5DE-4B64-BAEA-D4CD80F847A2@joetify.com> References: <486130C5-A5DE-4B64-BAEA-D4CD80F847A2@joetify.com> Message-ID: <20110322113754.GA2589@hanele.lan> # Joe Williams 2011-03-22: > Anyone else run into this issue? > > 1> file:consult("/usr/local/lib/erlang/lib/wx-0.98.9/ebin/wx.app"). > {error,{26,erl_parse, > ["syntax error before: ","wxAcceleratorTable"]}} > > At least on my system (OSX) it looks like the file contains lines like > "wxAcceleratorEntry,n wxAcceleratorTable,n wxArtProvider,n ...". > On my linux box it seems fine. That's because lib/wx/src/Makefile makes an unreasonable assumption that host OS's "sed" command happens to be the GNU implementation. As a workaround, change this rule: $(APP_TARGET): $(APP_SRC) ../vsn.mk Makefile sed -e 's;%GEN_MODS%;$(GEN_MODS);' $< > $@.tmp sed -e 's;%VSN%;$(VSN);' $@.tmp > $@ rm $@.tmp to use GNU sed instead. Alternatively, change GEN_MODS definition to: GEN_MODS = $(GEN_FILES:gen/%.erl=%,) (Surely you won't mind wx.app will end up looking less pretty than intended). Alternatively, just hand-edit wx.app changing ",n" to ",". HTH, -- Jachym From raimo+erlang-questions@REDACTED Tue Mar 22 14:04:44 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 22 Mar 2011 14:04:44 +0100 Subject: Mailing list software change Message-ID: <20110322130444.GE12691@erix.ericsson.se> Hi all. We will change servers and mailing list software from the old-fashion ezmlm to GNU Mailman on Thu Mar 24 afternoon (CET). All subscribtions will be transferred into the corresponding Mailman settings. The Mailman web interface is probably not up and running at first, we'll see about that until later. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From adam@REDACTED Tue Mar 22 15:35:49 2011 From: adam@REDACTED (Adam Lindberg) Date: Tue, 22 Mar 2011 15:35:49 +0100 Subject: Cover compilation is lossy Message-ID: <4D88B3C5.40300@erlang-solutions.com> Hi, I'm having a bit of a peculiar situation. In my mocking framework, meck, I want to enable mocking of cover compiled modules. I can do this in most situations, but there is one case where it doesn't work because of how cover compiles modules. If I have a module 'm', which have a function 'f' and compile that with [debug_info], I can get the original beam code (from disk) and the abstract code (from the module in memory). Both these can be used to restore the original module when done mocking (load the binary or recompile the abstract code). When cover compiling 'm' without any special flags, the same is true. But when using for example [{d, 'TEST'}] in cover, cover will compile the module accordingly, BUT it will not save the compile option in the module info. It will neither save the new abstract code that it used to compile the module. This makes it impossible to either a) restore the module from abstract code or b) recompile it using the options provided. So, how could I go about restoring the cover compiled module as seen in memory before mocking? * I can't restore the beam (since it isn't cover compiled). * I can't restore the abstract code (since it isn't "cover compiled"). * I can't recompile the original source (since the compile options are thrown away by cover). Is there any way to get to the module beam code or abstract code for a cover compiled module? Or is the only way to fix cover to actually save the correct abstract code / compile options? (Current meck source code can be found here: https://github.com/eproxus/meck/blob/master/src/meck.erl#L463 backup_original/1 and restore_original/2 are of particular interest) Cheers, Adam From norton@REDACTED Tue Mar 22 15:11:21 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 22 Mar 2011 23:11:21 +0900 Subject: [erlang-questions] Nested finite state machines - Existing Implementations? In-Reply-To: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> References: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> Message-ID: Ken - UBF provides support for defining and for implementing an FSM by contract. http://norton.github.com/ubf/ubf-user-guide.en.html#_ubf_b However, there is no support for nested state machines. - Joe N. > > On 22 Mar 2011, at 04:47, Ken Robinson wrote: > >> Hi All, >> I have a standard which I'm implementing which calls for nested state >> machines which have push and pop states and entry and exit actions. >> I'm having difficulty using gen_fsm to do this. I've been looking at >> plain_fsm by Ulf Wiger which looks promising. Before I reinvent the >> wheel, is there any existing fully fledged nested state machines out >> there? >> >> -- >> regards, >> Ken Robinson >> Mob +61438681120 >> Home +61738523767 >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > Joseph Wayne Norton norton@REDACTED From ericbmerritt@REDACTED Tue Mar 22 16:03:48 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 22 Mar 2011 10:03:48 -0500 Subject: [erlang-questions] how to create compile warning for deprecated functions? In-Reply-To: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> References: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> Message-ID: Coax, At the moment you can't. There is a deprecated attribute that you can add to modules, but the only thing that uses it is the xref tool as Niclas said. The deprecated list that the complier uses is a hard coded list that (I assume) is manually updated for each release. Hopefully, that will change in the future, but for now the options are to use xref. This deprecated flag is document in the xref documentation. Eric On Tue, Mar 22, 2011 at 4:16 AM, caox wrote: > Hi, > > ? ? ? ?I have some lib functions will be deprecated in next release of our repo. How could I add a warning during compiling for other developers just as the module regexp does: > > src/lib/email/add_phone.erl:23: Warning: regexp:split/2: the regexp module is deprecated (will be removed in R15A); use the re module instead > > regards, > caox > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From spawn.think@REDACTED Tue Mar 22 16:16:12 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 22 Mar 2011 16:16:12 +0100 Subject: [erlang-questions] how to create compile warning for deprecated functions? In-Reply-To: References: <0B607186-BDF2-47D8-BB62-1C31996CA41D@lightpole.net> Message-ID: True, it's hard coded, in otp_internal.erl under stdlib On Tue, Mar 22, 2011 at 4:03 PM, Eric Merritt wrote: > Coax, > > At the moment you can't. There is a deprecated attribute that you can > add to modules, but the only thing that uses it is the xref tool as > Niclas said. The deprecated list that the complier uses is a hard > coded list that (I assume) is manually updated for each release. > > Hopefully, that will change in the future, but for now the options are > to use xref. This deprecated flag is document in the xref > documentation. > > Eric > > On Tue, Mar 22, 2011 at 4:16 AM, caox wrote: > > Hi, > > > > I have some lib functions will be deprecated in next release of > our repo. How could I add a warning during compiling for other developers > just as the module regexp does: > > > > src/lib/email/add_phone.erl:23: Warning: regexp:split/2: the regexp > module is deprecated (will be removed in R15A); use the re module instead > > > > regards, > > caox > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think From juanjo@REDACTED Tue Mar 22 19:43:02 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 22 Mar 2011 15:43:02 -0300 Subject: [JOB] Software developer for web-based VoIP project in Buenos Aires, Argentina Message-ID: My company (Erlar) is looking for a full-time talented software developer for a web-based VoIP project. Candidates should be passionate, proactive, resourceful and able to learn the skills necessary to solve any problems related to the project. A thorough knowledge of Erlang and OTP is a big plus, but training will be provided if necessary for candidates that pass the selection process. This is a great opportunity for any developer willing to learn Erlang and work with a team software engineers with several years of experience developing highly available web-based systems. If you're interested, please send your r?sum? or CV to careers@REDACTED The company: Founded in 2010, Erlar provides Erlang software consulting, contracting, training and support services. Specialized in the design and development of massively concurrent, high availability systems. Requirements: - BS or MS in Computer Science, or equivalent experience - Excellent communication skills in English - Experience working with functional programming languages (preferably Erlang) Nice-to-have: - Experience with Erlang and OTP - Experience working with the Erlang open-source ecosystem (couchbeam, erlsom, erlydtl, mochiweb, gen_leader, etc.) - Experience with NoSQL databases, mainly CouchDB - Familiar with FreeSWITCH and SIP - Familiar with distributed version control systems (git) - Familiar with Linux system administration (especially Ubuntu) - Knowledge of web protocols - Knowledge of C and Lua Location: - Buenos Aires, Argentina - Partial telecommuting is possible From jesper.louis.andersen@REDACTED Tue Mar 22 22:04:09 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 22 Mar 2011 22:04:09 +0100 Subject: Getting the MTU of an IPv4 UDP socket? Message-ID: What is the easiest way to query MTUs for UDP sockets on Linux and Windows (One is ok, I can hack it for Windows). I know that you should probably carry out some PMTU discovery phase, but I am somewhat interested if you can query this state of a socket and how supported it is. Meanwhile, I'll cheat and assume the MTU is 1000 bytes :P -- J. From iostres@REDACTED Tue Mar 22 22:18:09 2011 From: iostres@REDACTED (Ivan Ostres) Date: Tue, 22 Mar 2011 22:18:09 +0100 Subject: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: References: Message-ID: On 22/3/11 10:04 PM, Jesper Louis Andersen wrote: > What is the easiest way to query MTUs for UDP sockets on Linux and > Windows (One is ok, I can hack it for Windows). I know that you should > probably carry out some PMTU discovery phase, but I am somewhat > interested if you can query this state of a socket and how supported > it is. > > Meanwhile, I'll cheat and assume the MTU is 1000 bytes :P Hello Jesper, MTU is defined on data-link layer, not on transport. If you don't want to do it with PMTU discovery, simplest way is to run "ifconfig" in shell, it shows MTU on OSX and it should do the same on Linux: en0: flags=8863 mtu 1500 I hope this helps. BR, Ivan Ostres From jesper.louis.andersen@REDACTED Tue Mar 22 22:21:13 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 22 Mar 2011 22:21:13 +0100 Subject: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: References: Message-ID: On Tue, Mar 22, 2011 at 22:18, Ivan Ostres wrote: > MTU is defined on data-link layer, not on transport. If you don't want to do > it with PMTU discovery, simplest way is to run "ifconfig" in shell, it shows > MTU on OSX and it should do the same on Linux: > > en0: flags=8863 mtu 1500 > That is one way :) Is there one which is less involved, heh. -- J. From michael.santos@REDACTED Tue Mar 22 22:47:08 2011 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 22 Mar 2011 17:47:08 -0400 Subject: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: References: Message-ID: <20110322214708.GA25812@ecn.lan> On Tue, Mar 22, 2011 at 10:04:09PM +0100, Jesper Louis Andersen wrote: > What is the easiest way to query MTUs for UDP sockets on Linux and > Windows (One is ok, I can hack it for Windows). I know that you should > probably carry out some PMTU discovery phase, but I am somewhat > interested if you can query this state of a socket and how supported > it is. > > Meanwhile, I'll cheat and assume the MTU is 1000 bytes :P You could check /proc or /sysfs on Linux. For example: 1> {ok,F} = file:open("/proc/sys/net/ipv6/conf/wlan0/mtu", [read,binary]). {ok,<0.34.0>} 2> file:read(F, 1024). {ok,<<"1500\n">>} 3> {ok,F} = file:open("/sys/class/net/wlan0/mtu", [read,binary]). {ok,<0.34.0>} 4> file:read(F, 1024). {ok,<<"1500\n">>} From freza@REDACTED Tue Mar 22 22:47:07 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 22 Mar 2011 21:47:07 +0000 Subject: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: References: Message-ID: <20110322214707.GA2368@hanele.lan> # Jesper Louis Andersen 2011-03-22: > On Tue, Mar 22, 2011 at 22:18, Ivan Ostres wrote: > > > MTU is defined on data-link layer, not on transport. If you don't want to do > > it with PMTU discovery, simplest way is to run "ifconfig" in shell, it shows > > MTU on OSX and it should do the same on Linux: > > > > en0: flags=8863 mtu 1500 > > That is one way :) Is there one which is less involved, heh. inet:ifget/N, but it seems you need to supply name of the underlying device too. HTH, -- Jachym From kenrobinsonster@REDACTED Wed Mar 23 03:47:25 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Wed, 23 Mar 2011 12:47:25 +1000 Subject: [erlang-questions] Nested finite state machines - Existing Implementations? In-Reply-To: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> References: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> Message-ID: Hi Ulf, Thanks for the reply. Much to digest. From what was said in the post and what you've said here I think I will try a separate process for each state with perhaps gproc as a lookup mechanism. I'll experiment with plain_fsm and gen_fsm. cheers, Ken. Looking through the On Tue, Mar 22, 2011 at 5:50 PM, Ulf Wiger wrote: > > Hi Ken, > > I don't think nested state machines are used much in the erlang world, > since it's generally considered detrimental to "complify" the FSM running > in a single process. > > OTOH, sometimes, you need to, in order to implement some behaviour. > One of the examples I've used in presentations was when implementing > a Diameter stack (rfc3588). It (arbitrarily) combines different FSMs into > a single thread, and this caused me to refactor the spec into multiple > cooperating FSMs (using plain_fsm). The price you pay is that it becomes > hard ?to check that your code actually matches the spec - even though it > became easier to see that the code did what I think was the intention? > > At AXD301 (a long, long time ago, it seems), we had the need for at least > two levels of nesting, for a call-control FSM and a transport FSM - one for > each call leg. We did this by making the call-control FSM an active process, > a la plain_fsm, and implementing the transport FSMs as gen_fsm-style > callbacks (but dispatched from within the CC FSM). We went through many > iterations, so it's quite likely that we eventually departed from this model > and implemented all FSMs as separate processes. > > And going for as many processes as the problem called for was one of the > driving forces behind gproc, which can serve as a index for locating the FSM > you need (based on ordered_set semantics and supporting QLC, you should > be able to efficiently select even a subset of multiple FSMs). > > http://github.com/esl/gproc > > In my presentation, "Death by Accidental Complexity", I use a parameterised > bare-bones event loop for FSM programming, which I also extend with a > "select vector" to support a form of selective receive in the FSM callbacks. > > Most of these things, and more (e.g. a link to ExAT) can be found in the following > thread: > > http://forum.trapexit.org/viewtopic.php?p=15811&sid=197b687b90d851e0b7ffdaa4f1465f8f > > If memory serves, Joel Reymont solved the problem to his own satisfaction in > the poker server, but my quick late-night googling failed to reveal a good link. > > Joel? > > BR, > Ulf W > > > On 22 Mar 2011, at 04:47, Ken Robinson wrote: > >> Hi All, >> I have a standard which I'm implementing which calls for nested state >> machines which have push and pop states and entry and exit actions. >> I'm having difficulty using gen_fsm to do this. I've been looking at >> plain_fsm by Ulf Wiger which looks promising. Before I reinvent the >> wheel, is there any existing fully fledged nested state machines out >> there? >> >> -- >> regards, >> Ken Robinson >> Mob +61438681120 >> Home +61738523767 >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From kenrobinsonster@REDACTED Wed Mar 23 03:56:46 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Wed, 23 Mar 2011 12:56:46 +1000 Subject: [erlang-questions] Nested finite state machines - Existing Implementations? In-Reply-To: References: <51583283-FB9E-42FF-AF08-65C44D7E6EB5@erlang-solutions.com> Message-ID: Sorry. I meant to say a separate process for each state machine. There would be no nested states in a process. Ken. On Wed, Mar 23, 2011 at 12:47 PM, Ken Robinson wrote: > Hi Ulf, > Thanks for the reply. Much to digest. From what was said in the post > and what you've said here I think I will try a separate process for > each state with perhaps gproc as a lookup mechanism. I'll experiment > with plain_fsm and gen_fsm. > > cheers, > Ken. > > Looking through the > On Tue, Mar 22, 2011 at 5:50 PM, Ulf Wiger > wrote: >> >> Hi Ken, >> >> I don't think nested state machines are used much in the erlang world, >> since it's generally considered detrimental to "complify" the FSM running >> in a single process. >> >> OTOH, sometimes, you need to, in order to implement some behaviour. >> One of the examples I've used in presentations was when implementing >> a Diameter stack (rfc3588). It (arbitrarily) combines different FSMs into >> a single thread, and this caused me to refactor the spec into multiple >> cooperating FSMs (using plain_fsm). The price you pay is that it becomes >> hard ?to check that your code actually matches the spec - even though it >> became easier to see that the code did what I think was the intention? >> >> At AXD301 (a long, long time ago, it seems), we had the need for at least >> two levels of nesting, for a call-control FSM and a transport FSM - one for >> each call leg. We did this by making the call-control FSM an active process, >> a la plain_fsm, and implementing the transport FSMs as gen_fsm-style >> callbacks (but dispatched from within the CC FSM). We went through many >> iterations, so it's quite likely that we eventually departed from this model >> and implemented all FSMs as separate processes. >> >> And going for as many processes as the problem called for was one of the >> driving forces behind gproc, which can serve as a index for locating the FSM >> you need (based on ordered_set semantics and supporting QLC, you should >> be able to efficiently select even a subset of multiple FSMs). >> >> http://github.com/esl/gproc >> >> In my presentation, "Death by Accidental Complexity", I use a parameterised >> bare-bones event loop for FSM programming, which I also extend with a >> "select vector" to support a form of selective receive in the FSM callbacks. >> >> Most of these things, and more (e.g. a link to ExAT) can be found in the following >> thread: >> >> http://forum.trapexit.org/viewtopic.php?p=15811&sid=197b687b90d851e0b7ffdaa4f1465f8f >> >> If memory serves, Joel Reymont solved the problem to his own satisfaction in >> the poker server, but my quick late-night googling failed to reveal a good link. >> >> Joel? >> >> BR, >> Ulf W >> >> >> On 22 Mar 2011, at 04:47, Ken Robinson wrote: >> >>> Hi All, >>> I have a standard which I'm implementing which calls for nested state >>> machines which have push and pop states and entry and exit actions. >>> I'm having difficulty using gen_fsm to do this. I've been looking at >>> plain_fsm by Ulf Wiger which looks promising. Before I reinvent the >>> wheel, is there any existing fully fledged nested state machines out >>> there? >>> >>> -- >>> regards, >>> Ken Robinson >>> Mob +61438681120 >>> Home +61738523767 >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> > > > > -- > regards, > Ken Robinson > Mob +61438681120 > Home +61738523767 > -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From ulf.wiger@REDACTED Wed Mar 23 08:49:54 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 23 Mar 2011 08:49:54 +0100 Subject: edown update - MarkEdoc added Message-ID: I recently pushed a nice little contribution to edown, called markedoc. http://github.com/esl/edown Edown renders Edoc markup as Markdown, in order to make the docs easy to read while browsing a Github repository. The easiest way to use it is with rebar, {deps, [{edown, ".*", {git, "git://github.com/esl/edown.git", "HEAD"}}]}. {edoc_opts, [{doclet, edown_doclet}, {src_path, ["src/"]}, {subpackages, true}]}. then ./rebar doc will generate .md files, including a doc/README.md Copying doc/README.md to the parent directory will break links in the module index. These can be manually fixed, or you can try to automate it using something like https://github.com/esl/run_eqc/blob/master/mk_readme.escript (I'm working on making that more generic, but that seems to be quite a step up in complexity). MarkEdoc goes the other way, converting Markdown to Edoc markup. The main use would be to maintain the README.md file as regular Markdown, and convert it to a proper overview.edoc using MarkEdoc. https://github.com/esl/edown/blob/master/bin/MARKEDOC-README.md Many thanks to Henning Dietrich for this contribution. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jean-sebastien.pedron@REDACTED Wed Mar 23 11:57:24 2011 From: jean-sebastien.pedron@REDACTED (=?ISO-8859-1?Q?Jean-S=E9bastien_P=E9dron?=) Date: Wed, 23 Mar 2011 11:57:24 +0100 Subject: [erlang-questions] Release packaging (on Debian) and live upgrade In-Reply-To: <20110322214502.GA19624@alumni.caltech.edu> References: <4D886A5B.1090405@dumbbell.fr> <20110322214502.GA19624@alumni.caltech.edu> Message-ID: <4D89D214.3010107@dumbbell.fr> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22.03.2011 22:45, Anthony Molinaro wrote: > Hi, > > I use a combination of > > http://code.google.com/p/erlrc/ > and > http://code.google.com/p/fwtemplates/ > > to accomplish something similar on debian in 2008 and currently on Centos. > > Fwtemplates supports building .debs and .rpms and creates app and appup files > during upgrade in an automagic fashion. erlrc integrates these into the > packaging system. > > Some old blog posts are here > > http://dukesoferl.blogspot.com/2008/02/erlang-and-automake.html > http://dukesoferl.blogspot.com/2008/06/im-in-ur-erlangz-upgrading-ur.html > http://dukesoferl.blogspot.com/2009/04/erlrc-erlang-factory-talk.html > http://dukesoferl.blogspot.com/2009/05/automatic-appup-file-generation.html > http://dukesoferl.blogspot.com/2009/05/automatic-app-file-generation.html > http://dukesoferl.blogspot.com/2009/12/erlrc-and-rpm.html I previously heard of erlrc but not fwtemplates. Thank you very much for the links! I'll study all this, especially the "appup" automatic generation which seems very interesting. - -- Jean-S?bastien P?dron Yakaz (http://www.yakaz.com/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2J0hQACgkQa+xGJsFYOlOXQwCgxtfea1c8DoPuE96MACebn3kH 7VwAoKRJw2tlsWPyAuSbhNw5vmenL2EI =uEGx -----END PGP SIGNATURE----- From goura@REDACTED Wed Mar 23 13:24:36 2011 From: goura@REDACTED (Kazuhiro Ogura) Date: Wed, 23 Mar 2011 21:24:36 +0900 Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem In-Reply-To: References: <4D81E7BB.8090802@gmail.com> <311465162.73841300359543943.JavaMail.root@zimbra> Message-ID: It was CentOS 5.5 (on vmware). K.Ogura 2011/3/21 Boris M?hmer : > Could You please say, which OS You use? > > I reported this "strange behavior" for Ubuntu 10.04 / 10.10 some time ago. > > > ?- boris > > > 2011/3/21 Kazuhiro Ogura : >> Hi, >> >> I had the same error, and after some struggles, >> finally I changed the shebang line of >> lib/erl_docgen/priv/bin/xref_mod_app.escript >> to the absolute path to escript, and it worked. >> >> -#!/usr/bin/env escript >> +#!/usr/local/bin/escript >> >> I don't know why it worked, >> so I can't tell whether this works for you or not. >> >> Sure I have xmlerl-x.y.z in the lib/ dir, and I can load xmerl in a >> normal erlang shell. >> Also what's strange is that when I manually execute the command, >> (which fails complaining about exception about xmerl:export_simple/2 >> in the make process), >> it succeeds without any errors. >> >> (May be some envvar passed to script is doing something wrong?) >> >> Kazuhiro Ogura >> >> 2011/3/17 Lukas Larsson : >>> It seems to me like xmerl is not in the erlang install which is in your path. Check that xmerl-x.y.z is in the lib/ dir of the erlang you have in your path. or alternatively that you can load the xmerl module in a normal erlang shell. >>> >>> Lukas >>> >>> ----- Original Message ----- >>> From: "Damian Dobroczy?ski" >>> To: erlang-questions@REDACTED >>> Sent: Thursday, 17 March, 2011 11:51:39 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna >>> Subject: [erlang-questions] Erlang/OTP R14B02 generating documentation from source problem >>> >>> >>> Hi. >>> >>> I downloaded, compiled and tried to "make docs" and "make install-docs". >>> The first went fine. The result of the latter is the following: >>> >>> escript: exception error: undefined function xmerl:export_simple/2 >>> ?in function ?erl_eval:do_apply/5 >>> ?in call from erl_eval:expr/5 >>> ?in call from erl_eval:exprs/5 >>> ?in call from erl_eval:local_func/5 >>> ?in call from escript:interpret/4 >>> ?in call from escript:start/1 >>> ?in call from init:start_it/1 >>> >>> So, I installed without docs ("make install") and then tried again. The >>> result was the same. So I removed all escript links and files that I had >>> had already (some artifacts from previous releases and such) so that I >>> had only fresh escript in the installation directory and a link in the >>> prefix (/usr/local/bin) directory. The result was the same. Can somebody >>> help me, please? >>> >>> D. >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > From andrew@REDACTED Wed Mar 23 17:21:27 2011 From: andrew@REDACTED (Andrew Thompson) Date: Wed, 23 Mar 2011 12:21:27 -0400 Subject: [erlang-questions] edown update - MarkEdoc added In-Reply-To: References: Message-ID: <20110323162127.GE20461@hijacked.us> On Wed, Mar 23, 2011 at 08:49:54AM +0100, Ulf Wiger wrote: > > I recently pushed a nice little contribution to edown, called markedoc. > > http://github.com/esl/edown > > Edown renders Edoc markup as Markdown, in order to make the docs easy > to read while browsing a Github repository. > > The easiest way to use it is with rebar, > > {deps, [{edown, ".*", {git, "git://github.com/esl/edown.git", "HEAD"}}]}. > {edoc_opts, [{doclet, edown_doclet}, > {src_path, ["src/"]}, > {subpackages, true}]}. > I tried to add this to gen_smtp (https://github.com/Vagabond/gen_smtp) and it blows up with: ==> gen_smtp (doc) edoc: error in layout 'edown_layout': {'EXIT',{function_clause,[{edown_layout,t_type,[[{xmlElement,nonempty_list,[],[],{xmlNamespace,[],[]},[{type,2},{argtypes,...},{...}|...],1,[],[...],...}]]},{edown_layout,t_utype_elem,1},{edown_layout,seq,4},{edown_layout,seq,4},{edown_layout,t_fun,1},{edown_layout,typespec,1},{edown_layout,function,2},{lists,flatmap,2}]}}. edoc: error in doclet 'edown_doclet': {'EXIT',error}. ERROR: doc failed while processing /Users/andrew/gen_smtp: {'EXIT',error} So I'm assuming I've got some edge case you didn't consider. Thanks, Andrew From ulf.wiger@REDACTED Wed Mar 23 17:27:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 23 Mar 2011 17:27:26 +0100 Subject: [erlang-questions] edown update - MarkEdoc added In-Reply-To: <20110323162127.GE20461@hijacked.us> References: <20110323162127.GE20461@hijacked.us> Message-ID: On 23 Mar 2011, at 17:21, Andrew Thompson wrote: > I tried to add this to gen_smtp (https://github.com/Vagabond/gen_smtp) > and it blows up with: > > [interesting blowup snipped] > > So I'm assuming I've got some edge case you didn't consider. Looks like it. Thanks for letting me know. I'll look into it. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Wed Mar 23 17:40:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 23 Mar 2011 17:40:26 +0100 Subject: [erlang-questions] edown update - MarkEdoc added In-Reply-To: References: <20110323162127.GE20461@hijacked.us> Message-ID: <71CA5C2F-FDF3-428C-BF00-6BC9756461F4@erlang-solutions.com> I cloned gen_smtp, added the edown deps and options to rebar.config and called ./rebar compile doc It didn't blow up. Using OTP R14B. diff --git a/rebar.config b/rebar.config index 05d2462..8f05e33 100644 --- a/rebar.config +++ b/rebar.config @@ -3,3 +3,5 @@ {erl_opts, [fail_on_warning, debug_info, warn_unused_vars, warn_unused_import, warn_ {cover_enabled, true}. {cover_print_enabled, true}. +{deps, [{edown, ".*", {git, "git://github.com/esl/edown.git", "HEAD"}}]}. +{edoc_opts, [{doclet, edown_doclet}]}. (I did try with the added options given in my original post, but still no crash.) I'm not sure what we're doing differently, but I couldn't reproduce your problem. Having said that, I'm sure there *are* corner cases where edown needs work, so I'll happily receive more examples of where it falls short so I can improve it. BR, Ulf W On 23 Mar 2011, at 17:27, Ulf Wiger wrote: > > On 23 Mar 2011, at 17:21, Andrew Thompson wrote: > >> I tried to add this to gen_smtp (https://github.com/Vagabond/gen_smtp) >> and it blows up with: >> >> [interesting blowup snipped] >> >> So I'm assuming I've got some edge case you didn't consider. > > Looks like it. Thanks for letting me know. I'll look into it. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From andrew@REDACTED Wed Mar 23 17:44:57 2011 From: andrew@REDACTED (Andrew Thompson) Date: Wed, 23 Mar 2011 12:44:57 -0400 Subject: [erlang-questions] edown update - MarkEdoc added In-Reply-To: <71CA5C2F-FDF3-428C-BF00-6BC9756461F4@erlang-solutions.com> References: <20110323162127.GE20461@hijacked.us> <71CA5C2F-FDF3-428C-BF00-6BC9756461F4@erlang-solutions.com> Message-ID: <20110323164456.GF20461@hijacked.us> On Wed, Mar 23, 2011 at 05:40:26PM +0100, Ulf Wiger wrote: > > I cloned gen_smtp, added the edown deps and options to rebar.config and called ./rebar compile doc > > It didn't blow up. Using OTP R14B. > I'm using R14B02, maybe that's the issue? edoc grew support for reading -specs. Andrew From anthonym@REDACTED Wed Mar 23 18:26:06 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Wed, 23 Mar 2011 10:26:06 -0700 Subject: [erlang-questions] Release packaging (on Debian) and live upgrade In-Reply-To: <4D89D214.3010107@dumbbell.fr> References: <4D886A5B.1090405@dumbbell.fr> <20110322214502.GA19624@alumni.caltech.edu> <4D89D214.3010107@dumbbell.fr> Message-ID: <20110323172606.GB22829@alumni.caltech.edu> Also available but not really released yet (in that I've use it for a few things but not enough to warrant me calling it done), is a framewerk template which uses rebar, and can be used to wrap and package things from github which use rebar. For instance I wanted proper from github so I did % fw-init --name proper --template erlang-rebar --revision svn \ --svn_project_path https://svn-priv.openx.org/svn/trunk/vendor/erlang/proper \ --svn_tag_root https://svn-priv.openx.org/svn/tags/vendor/erlang/proper \ --wrap_git_path git://github.com/manopapad/proper.git % cd proper % ./bootstrap && ./configure && make package % rpm -ivh fw-pkgout/proper-0.0.0-TEST1.noarch.rpm Which creates a framewerk package in my svn directory which is a wrapper package for a github tree which will compile with rebar. After the make package command I have an rpm which will install proper into the erlang lib dir on the machine. Also, 'make release' publishes the rpm to my yum repo (in the case of .deb packaging 'release' puts the package onto an apt repo). In addition you can always 'make dist' to get a self contained .tar.gz file which you can provide to people to install your erlang package with './configure && make && sudo make install'. The automatic app/appup stuff is separable in case you just need that stuff. -Anthony On Wed, Mar 23, 2011 at 11:57:24AM +0100, Jean-S?bastien P?dron wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 22.03.2011 22:45, Anthony Molinaro wrote: > > Hi, > > > > I use a combination of > > > > http://code.google.com/p/erlrc/ > > and > > http://code.google.com/p/fwtemplates/ > > > > to accomplish something similar on debian in 2008 and currently on Centos. > > > > Fwtemplates supports building .debs and .rpms and creates app and appup files > > during upgrade in an automagic fashion. erlrc integrates these into the > > packaging system. > > > > Some old blog posts are here > > > > http://dukesoferl.blogspot.com/2008/02/erlang-and-automake.html > > http://dukesoferl.blogspot.com/2008/06/im-in-ur-erlangz-upgrading-ur.html > > http://dukesoferl.blogspot.com/2009/04/erlrc-erlang-factory-talk.html > > http://dukesoferl.blogspot.com/2009/05/automatic-appup-file-generation.html > > http://dukesoferl.blogspot.com/2009/05/automatic-app-file-generation.html > > http://dukesoferl.blogspot.com/2009/12/erlrc-and-rpm.html > > I previously heard of erlrc but not fwtemplates. Thank you very much for > the links! > > I'll study all this, especially the "appup" automatic generation which > seems very interesting. > > - -- > Jean-S?bastien P?dron > Yakaz (http://www.yakaz.com/) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.17 (FreeBSD) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk2J0hQACgkQa+xGJsFYOlOXQwCgxtfea1c8DoPuE96MACebn3kH > 7VwAoKRJw2tlsWPyAuSbhNw5vmenL2EI > =uEGx > -----END PGP SIGNATURE----- > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From vances@REDACTED Wed Mar 23 18:37:33 2011 From: vances@REDACTED (Vance Shipley) Date: Wed, 23 Mar 2011 13:37:33 -0400 Subject: [ANN] Radierl v1.0: RADIUS Protocol Stack Message-ID: <20110323173652.GA6861@h216-235-12-173.host.egate.net> The radierl project implements the RADIUS protocol, as decribed in RFC2865 and RFC2866, as a stack application in Erlang/OTP. It includes test suites, documentation and an example application which implements basic athentication and accounting servers using mnesia and disk_log respectively. The project is hosted here: https://github.com/vances/radierl -- -Vance From fjl@REDACTED Wed Mar 23 21:56:05 2011 From: fjl@REDACTED (Felix Lange) Date: Wed, 23 Mar 2011 13:56:05 -0700 Subject: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: <20110322214707.GA2368@hanele.lan> References: <20110322214707.GA2368@hanele.lan> Message-ID: <9B617ABD-6846-4CEC-9DEE-A64BD303F879@twurst.com> You can use inet:getiflist/0 to get a list of all network interfaces. Example (on my machine): 24> inet:getiflist(). {ok,["lo0","en0"]} 25> inet:ifget("en0", [mtu]). {ok,[{mtu,1500}]} On 22 Mar 2011, at 14:47, Jachym Holecek wrote: > # Jesper Louis Andersen 2011-03-22: >> On Tue, Mar 22, 2011 at 22:18, Ivan Ostres wrote: >> >>> MTU is defined on data-link layer, not on transport. If you don't want to do >>> it with PMTU discovery, simplest way is to run "ifconfig" in shell, it shows >>> MTU on OSX and it should do the same on Linux: >>> >>> en0: flags=8863 mtu 1500 >> >> That is one way :) Is there one which is less involved, heh. > > inet:ifget/N, but it seems you need to supply name of the underlying device too. > > HTH, > -- Jachym > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From uaforum1@REDACTED Thu Mar 24 09:15:33 2011 From: uaforum1@REDACTED (u a) Date: Thu, 24 Mar 2011 01:15:33 -0700 (PDT) Subject: Webmachine questions Message-ID: <3572825e-7f50-4f54-ad0c-337dada29c79@r13g2000yqk.googlegroups.com> Hello, i am working on a small project, where i am using webmachine. Everything works fine, but know i have some question regarding the tool. 1. Tunneld delete throw POST (HTML forms) Because of the limitation of the browser, i have to tunnel a delete request throw a post request to webmachine. When i look in the trace, great tool :-) , my request comes to the decision M16 where the question is asked if this is a DELETE method or not. The answer is false , because it is a POST and not a DELETE method. Is it possible to change the method from a POST to a DELETE or which kind of solution is the best? 2. Getting the protocol for the location When i create a location header or a redirect, i want to decide of the underlying socket, which kind of protocol i have to use (HTTP or HTTPS for instance. I found that the socket is part of webmachine_request, but there is no function inside wrq to retrieve this information. Is there an easy way to detect the protocol which is used inside the resource? Thanks for your help, Ulf From alain.odea@REDACTED Thu Mar 24 16:48:13 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Thu, 24 Mar 2011 13:18:13 -0230 Subject: [erlang-questions 1] Re: [erlang-questions] Webmachine questions In-Reply-To: <3572825e-7f50-4f54-ad0c-337dada29c79@r13g2000yqk.googlegroups.com> References: <3572825e-7f50-4f54-ad0c-337dada29c79@r13g2000yqk.googlegroups.com> Message-ID: <5206FCBE-8FF4-464F-8464-31C468926C2E@gmail.com> Hi Ulf: On 2011-03-24, at 5:45, u a wrote: > Hello, > > i am working on a small project, where i am using webmachine. > Everything works fine, but know i have some question regarding the > tool. > > 1. Tunneld delete throw POST (HTML forms) > > Because of the limitation of the browser, i have to tunnel a delete > request throw a post request to webmachine. > When i look in the trace, great tool :-) , my request comes to the > decision M16 where the question is asked if this is a DELETE method or > not. > The answer is false , because it is a POST and not a DELETE method. > > Is it possible to change the method from a POST to a DELETE or which > kind of solution is the best? > > 2. Getting the protocol for the location > > When i create a location header or a redirect, i want to decide of the > underlying socket, which kind of protocol i have to use (HTTP or HTTPS > for instance. I found that the socket is part of webmachine_request, > but there is no function inside wrq to retrieve this information. > > Is there an easy way to detect the protocol which is used inside the > resource? > > Thanks for your help, > Ulf > 1. Your resource's process_post function needs to handle this. Since you are tunneling DELETE through POST it is actually POST that you need to handle. I am guessing your tunneling is based on a form param called method. Read that and branch in process_post :) 2. No idea off the top of my head how to get the protocol. I don't think vanilla webmachine wrq has this info. I'm pretty sure it is visible in Zotonic webmachine resources though From roberto@REDACTED Thu Mar 24 21:27:16 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 24 Mar 2011 21:27:16 +0100 Subject: [erlang-questions 2] dialyzer and ssl Message-ID: dear list, i'm getting this dialyzer warning i cannot understand: "The pattern 'ok' can never match the type {'error',_} | {'ok',{'sslsocket',_,_}}" this points to line 4 of this code extract: 1. case ssl:transport_accept(ListenSocket) of 2. {ok, {sslsocket, _, _} = Sock} -> 3. case ssl:ssl_accept(Sock, 60000) of 4. ok -> ... 5. {error, _Reason} -> ... 6. end 7. {error, _Reason} -> ... 8. end afaik, ssl_accept/2 returns ok | {error, Reason} as per the specs: http://www.erlang.org/doc/man/ssl.html#ssl_accept-2 is this due to some discrepancy in docs/specs? or am i missing something? thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Mar 25 00:05:11 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 25 Mar 2011 00:05:11 +0100 Subject: [erlang-questions 3] Re: [erlang-questions] Getting the MTU of an IPv4 UDP socket? In-Reply-To: <9B617ABD-6846-4CEC-9DEE-A64BD303F879@twurst.com> References: <20110322214707.GA2368@hanele.lan> <9B617ABD-6846-4CEC-9DEE-A64BD303F879@twurst.com> Message-ID: On Wed, Mar 23, 2011 at 21:56, Felix Lange wrote: > > You can use inet:getiflist/0 to get a list of all network interfaces. > Example (on my machine): [..] Neat! I hadn't really dug deep enough in the modules, I see. -- J. From apauley@REDACTED Fri Mar 25 13:15:16 2011 From: apauley@REDACTED (Andreas Pauley) Date: Fri, 25 Mar 2011 14:15:16 +0200 Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python Message-ID: Hi all, In my quest to learn Erlang I've translated Peter Norvig's sudoku solver into Erlang: https://github.com/apauley/sudoku-in-erlang For comparison, my slightly modified version of Norvig's Python code can be found here: https://github.com/apauley/sudoku-by-norvig I like the pattern matching, it was fun to do the entire solution without a single if or case statement :-) Something that bothers me though is that the Python solution seems to be faster, even after I've made the Erlang solution use multiple processors. In order to compare the two solutions I counted the number of eliminations each performed. The eliminate function is the core of the solution, for example assigning a single value to a square is implemented as the elimination of all other values. With the Erlang implementation I get: sudoku-in-erlang$ ./sudoku All tests passed :-) Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). And with the Python implementation: sudoku-by-norvig$ ./sudoku.py All tests pass. Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) (33059 total eliminations, avg 661.00, median 648, max 830, min 648). Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). So according to the stats above, the Python solution performs less computations when given exactly the same input. The Erlang code is as close to the Python as I could make it, I've done more or less a direct translation of the algorithms used. I suspect that there are some lazy evaluation happening in the Python version, possibly generators, although I haven't pinpointed it yet. How can I improve my Erlang code in this solution? Kind regards, Andreas From bomicael@REDACTED Fri Mar 25 14:15:49 2011 From: bomicael@REDACTED (Micael Karlberg) Date: Fri, 25 Mar 2011 14:15:49 +0100 Subject: [erlang-questions 5] Testing Message-ID: Testing. /BMK From fdmanana@REDACTED Fri Mar 25 15:56:14 2011 From: fdmanana@REDACTED (Filipe David Manana) Date: Fri, 25 Mar 2011 14:56:14 +0000 Subject: [erlang-questions 6] enif_get_double Message-ID: Is there any equivalent in R13B03? My initial though was through C pre processor macros, test the OTP major and minor release macros, and if it's R13B03 to define this function as it defined in the R13B04 source (erl_nif.c), however it depends on a few OTP internals (which depend on other ones, etc) not exported - i know, this sounds too hackish, but was wondering if I could make a NIF run in R13B03, R13B04 and R14 by playing with C pre processor macros. regards -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From maxtqm@REDACTED Fri Mar 25 17:37:21 2011 From: maxtqm@REDACTED (max tan) Date: Sat, 26 Mar 2011 00:37:21 +0800 Subject: [erlang-questions 7] Why bit syntax matching can't be used in match_spec? Message-ID: It seems bit syntax matching can't be used in match_spec, I just wonder why. in erlang shell: ............................. 39> mnesia:dirty_select(main_dir, [{{k_v, <<"/root"/binary, "_"/binary>>, '_'}, [], ['$_']}]). ** exception error: bad argument .............................. 47> ets:fun2ms(fun ({k_v, <<"/root", Res/binary>>, _}) -> Res end). Error: fun head contains bit syntax matching of variable 'Res', which cannot be translated into match_spec {error,transform_error} 48> ............................. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Fri Mar 25 18:23:01 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Fri, 25 Mar 2011 18:23:01 +0100 Subject: [erlang-questions 8] Re: [erlang-questions] code_server:call/2 problem? In-Reply-To: Message-ID: Hi, By the way, looking at the article about Dialyzer's race condition detection capabilities (http://www.it.uu.se/research/group/hipe/dialyzer/publications/races.pdf), I was surprised that what appears like a race condition involving code_server was not spotted, whereas it seemed to correspond to the process registry-based race condition type described in the article (first documented example). Unless it corresponds to one of the 4 "ProcR" conditions established for kernel, and it has not been fixed yet? Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 spawn.think@REDACTED Envoy? par : erlang-questions@REDACTED 21/03/2011 14:46 A olivier.boudeville@REDACTED cc mevans@REDACTED, erlang-questions@REDACTED Objet Re: [erlang-questions] code_server:call/2 problem? Hi Olivier, I think it has to do with the supervisor children order in kernel app. currently the rpc server is started before the code_server, i'm not sure if there's a good reason for that {ok, {SupFlags, [Rpc, Global, InetDb | DistAC] ++ [NetSup, Glo_grp, File, Code, StdError, User, Config, SafeSupervisor] ++ Timer}} On Mon, Mar 21, 2011 at 1:22 PM, Olivier BOUDEVILLE < olivier.boudeville@REDACTED> wrote: > Hi, > > Thanks for your answer. Indeed, this could have been explanation; however > the node is crashing after that error, not before nor "in parallel" to the > error. > > Actually I believe there is a bug in the Erlang runtime. I strongly > suspect there is a small time window during which a race condition can > occur: apparently code:load_binary can be triggered (thanks to > rpc:multicall) on a just-launched node before at least one of its system > processes succeeds in registering its name. At least that's what I came to > think after having peered at lib/kernel/src/code_server.erl, the badarg > that occured may come from the fact that call/2 is called whereas Name is > not registered (yet), in: > > """ > call(Name, Req) -> > Name ! {code_call, self(), Req}, > receive > {?MODULE, Reply} -> > Reply > end. > """" > > As a test, the non-systematic crash which, on our short test case (done on > Ubuntu 64-bit running on a 4--core Core i7 laptop), was taking on average > 30 seconds (loop of ~15 attempts) before happening, never happened with > the same loop being run for more than one hour, once I inserted a > timer:sleep(1000) in my deployment manager between the launching of the > remote VM and the call to rpc:multicall (knowing that intermediate > checkings like Erlang ping of the remote node and checking of the remote > Erlang version always succeeded). > > I suppose there is in the runtime a kind of synchronous barrier where all > system processes are checked to be up and ready (including appropriately > registered) before serving user-space requests, but probably that at least > one system process was forgotten and thus led to such a race condition. > Unless I am mistaken? > > Thanks in advance for any answer, > Best regards, > > Olivier. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > mevans@REDACTED > Envoy? par : erlang-questions@REDACTED > 18/03/2011 19:22 > > A > olivier.boudeville@REDACTED, erlang-questions@REDACTED > cc > > Objet > RE: [erlang-questions] code_server:call/2 problem? > > > > > > > Do you see any crashes on the remote nodes? > > It does look like the remote code_server application has got a request, > but it for some reason fails (corrupted data perhaps?). I'm wondering if > you could attach a remote shell to one of those nodes and trace the > code_server module? > > -----Original Message----- > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On > Behalf Of Olivier BOUDEVILLE > Sent: Friday, March 18, 2011 1:32 PM > To: erlang-questions@REDACTED > Subject: [erlang-questions] code_server:call/2 problem? > > Hi, > > We are running a distributed Erlang program on a user node from which a > number of computing nodes are spawned, via SSH for the remote hosts. To > perform the automatic deployment, two deployment-related modules are sent > to each of the spawned nodes, using the traditional approach (first a call > > to code:get_object_code/1 then a rpc:multicall of code:load_binary). > > However, sometimes (not frequently), with the exact same settings, the > first module cannot be deployed successfully. We have indeed: > > {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ > ModuleName, ModuleFilename, ModuleBinary ], Timeout ), > > that returns: > ResList = > > [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] > BadNodes = [] > > This happens with R14B02, but most probably with previous versions as > well. > Apparently this happens often (always?) on a node created on the user > host. > I am pretty sure the deployed node is "fresh" (blank, vanilla). > And ignoring the badrpc will result in a undef error as soon as the first > function of the first helper module is called, even if delaying the call > (a race condition was suspected if ever the actual loading was > asynchronous). > > Would anyone see a cause for such a badarg non-systematic error? > > Thanks in advance for any hint, > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui > y figurent sont strictement confidentielles. Toute utilisation de ce > Message non conforme ? sa destination, toute diffusion ou toute > publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en > avertir imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete > it and all copies from your system and notify the sender immediately by > return message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui y > figurent sont strictement confidentielles. Toute utilisation de ce Message > non conforme ? sa destination, toute diffusion ou toute publication totale > ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete it > and all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwilberding@REDACTED Fri Mar 25 19:16:38 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Fri, 25 Mar 2011 14:16:38 -0400 Subject: [erlang-questions 9] Dead links on new site Message-ID: I couldn't immediately find a contact for the erlang.org website, so just going to post here. There is a dead link for "entire source code" and "EPL translation "at http://www.erlang.org/about.html Thanks! Jordan Wilberding -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Mar 25 19:33:03 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 25 Mar 2011 19:33:03 +0100 Subject: [erlang-questions 10] Re: Dead links on new site In-Reply-To: References: Message-ID: <20110325183303.GA10692@erix.ericsson.se> On Fri, Mar 25, 2011 at 02:16:38PM -0400, Jordan Wilberding wrote: > I couldn't immediately find a contact for the erlang.org website, so just > going to post here. > > There is a dead link for "entire source > code" and > "EPL translation "at > http://www.erlang.org/about.html > > Thanks! > Jordan Wilberding The de-facto standard webmaster@REDACTED should work, but I will add a contacts link somewhere, and check your findings... Thank you for reporting this! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From michael.santos@REDACTED Fri Mar 25 19:50:08 2011 From: michael.santos@REDACTED (Michael Santos) Date: Fri, 25 Mar 2011 14:50:08 -0400 Subject: [erlang-questions 11] fun with the new erlang.org! Message-ID: <20110325185008.GA32567@ecn.lan> http://www.erlang.org/ And the WINNAR! http://www.erlang.org/</div><div><h1 style="font-size:60px;">Powered by node.js</h1></div> ;) From anders.nygren@REDACTED Fri Mar 25 19:52:53 2011 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 25 Mar 2011 12:52:53 -0600 Subject: [erlang-questions 12] Re: dialyzer and ssl In-Reply-To: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> References: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> Message-ID: <AANLkTimkauACTQJaRjKKktAGoJZGOyPWsJiFTLBbQPCn@mail.gmail.com> According to the documentation transport_accept/2 returns {ok, NewSocket} | {error, Reason} Which is what dialyzer also claims. So Your case clause on line 4 can never match. /Anders On Thu, Mar 24, 2011 at 2:27 PM, Roberto Ostinelli <roberto@REDACTED> wrote: > dear list, > > i'm getting this dialyzer warning i cannot understand: > > "The pattern 'ok' can never match the type {'error',_} | > {'ok',{'sslsocket',_,_}}" > > this points to line 4 of this code extract: > > 1. case ssl:transport_accept(ListenSocket) of > 2. ?? {ok, {sslsocket, _, _} = Sock} -> > 3. ?? ??? case ssl:ssl_accept(Sock, 60000) of > 4. ?? ??? ??? ok -> > ??? ??? ??? ??? ... > 5. ?? ??? ??? {error, _Reason} -> > ??? ??? ??? ??? ... > 6. ?? ??? end > 7. ?? {error, _Reason} -> > ??? ??? ... > 8. end > > afaik, ssl_accept/2 returns ok | {error, Reason} as per the specs: > http://www.erlang.org/doc/man/ssl.html#ssl_accept-2 > > is this due to some discrepancy in docs/specs? or am i missing something? > > thank you, > > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From alain.odea@REDACTED Fri Mar 25 21:00:35 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 25 Mar 2011 17:30:35 -0230 Subject: [erlang-questions 13] Re: [erlang-questions] Webmachine questions In-Reply-To: <AANLkTin2iShmS0gDqWHCCUneO5sLudfMA5+GM0H4TDBX@mail.gmail.com> References: <3572825e-7f50-4f54-ad0c-337dada29c79@r13g2000yqk.googlegroups.com> <5206FCBE-8FF4-464F-8464-31C468926C2E@gmail.com> <AANLkTin2iShmS0gDqWHCCUneO5sLudfMA5+GM0H4TDBX@mail.gmail.com> Message-ID: <AANLkTi=zOFb5vfNYuWONdnD5-Uy9RzwCku9pykwcrZdx@mail.gmail.com> Hi Ulf: You won't actually need two delete functions, since process_post can delegate to your delete handler if the correct form parameter is present. It's a compromise. Modifying the webmachine_decision_core to handle protocol tunneling would be a serious challenge. An alternative would be to have to have an HTTP proxy in front of webmachine that understands your representation of tunneling. Overall a proxy would be epically more complicated and operational problematic than a small amount of branching within a resource. A much better alternative is to implement a generic webmachine resource that accepts a callback module whose functions perform the application-level request processing. That way you could effectively have a generic HTTP method tunneling webmachine resource :) None of this is necessary if you use XHR to send your forms since it allows you to set the method. Cheers, Alain On Fri, Mar 25, 2011 at 1:58 AM, Ulf Angermann <uangermann@REDACTED> wrote: > Hi Alan, > > thanks for answer, but i am not really happy with your first one, because > than i have to implement two "delete" functions. In my opinion it would be > nicer, if i could change the method from POST to DELETE. Than i could handle > the tunneld delete as the "normal" one. > > I will look at zotonic if i find something there, thank you. > > Thanks, > Ulf > > 2011/3/24 Alain O'Dea <alain.odea@REDACTED> >> >> Hi Ulf: >> >> On 2011-03-24, at 5:45, u a <uaforum1@REDACTED> wrote: >> >> > Hello, >> > >> > i am working on a small project, where i am using webmachine. >> > Everything works fine, but know i have some question regarding the >> > tool. >> > >> > 1. Tunneld delete throw POST (HTML forms) >> > >> > Because of the limitation of the browser, i have to tunnel a delete >> > request throw a post request to webmachine. >> > When i look in the trace, great tool :-) , my request comes to the >> > decision M16 where the question is asked if this is a DELETE method or >> > not. >> > The answer is false , because it is a POST and not a DELETE method. >> > >> > Is it possible to change the method from a POST to a DELETE or which >> > kind of solution is the best? >> > >> > 2. Getting the protocol for the location >> > >> > When i create a location header or a redirect, i want to decide of the >> > underlying socket, which kind of protocol i have to use (HTTP or HTTPS >> > for instance. I found that the socket is part of webmachine_request, >> > but there is no function inside wrq to retrieve this information. >> > >> > Is there an easy way to detect the protocol which is used inside the >> > resource? >> > >> > Thanks for your help, >> > Ulf >> > >> >> 1. Your resource's process_post function needs to handle this. ?Since you >> are tunneling DELETE through POST it is actually POST that you need to >> handle. ?I am guessing your tunneling is based on a form param called >> method. ?Read that and branch in process_post :) >> >> 2. No idea off the top of my head how to get the protocol. ?I don't think >> vanilla webmachine wrq has this info. ?I'm pretty sure it is visible in >> Zotonic webmachine resources though > From antoine.koener@REDACTED Fri Mar 25 22:18:15 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Fri, 25 Mar 2011 22:18:15 +0100 Subject: [erlang-questions 14] New Mailing list software Message-ID: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> Hi, Every single mail seems to be seen as Junk now in Apple Mail. Is this just me ? :) PS: I know that there's many other imap clients out there :-) From fred.hebert@REDACTED Fri Mar 25 22:22:50 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Fri, 25 Mar 2011 14:22:50 -0700 Subject: [erlang-questions 15] Re: New Mailing list software In-Reply-To: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> Message-ID: <56F659B0-8538-47C2-85EA-7DA768050683@erlang-solutions.com> No junk for me, although I worry about the XSS issue pointed earlier. -- Fred H?bert http://www.erlang-solutions.com On 2011-03-25, at 14:18 PM, Koener Antoine wrote: > Hi, > > Every single mail seems to be seen as Junk now in Apple Mail. > > Is this just me ? :) > > > PS: I know that there's many other imap clients out there :-) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From toby@REDACTED Fri Mar 25 22:34:22 2011 From: toby@REDACTED (Toby Thain) Date: Fri, 25 Mar 2011 17:34:22 -0400 Subject: [erlang-questions 16] Re: New Mailing list software In-Reply-To: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> Message-ID: <4D8D0A5E.3000208@telegraphics.com.au> On 25/03/11 5:18 PM, Koener Antoine wrote: > Hi, > > Every single mail seems to be seen as Junk now in Apple Mail. > > Is this just me ? :) Yes, it's just you; that is a client side function. --T > > > PS: I know that there's many other imap clients out there :-) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From s.j.thompson@REDACTED Fri Mar 25 22:44:54 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Fri, 25 Mar 2011 14:44:54 -0700 Subject: [erlang-questions 17] Re: New Mailing list software In-Reply-To: <4D8D0A5E.3000208@telegraphics.com.au> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <4D8D0A5E.3000208@telegraphics.com.au> Message-ID: <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> Something peculiar happening with header padding for me: getting ints in headers so my automated filing is not spotting them ... (this filing done before hitting client) Simon Sent from my iPhone On 25 Mar 2011, at 14:34, Toby Thain <toby@REDACTED> wrote: > On 25/03/11 5:18 PM, Koener Antoine wrote: >> Hi, >> >> Every single mail seems to be seen as Junk now in Apple Mail. >> >> Is this just me ? :) > > Yes, it's just you; that is a client side function. > > --T > > >> >> >> PS: I know that there's many other imap clients out there :-) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mevans@REDACTED Fri Mar 25 23:16:53 2011 From: mevans@REDACTED (Evans, Matthew) Date: Fri, 25 Mar 2011 18:16:53 -0400 Subject: [erlang-questions 18] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): Without native set: 18> c(sudoku). {ok,sudoku} 19> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok With native set: 20> c(sudoku,[native]). {ok,sudoku} 21> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas Pauley Sent: Friday, March 25, 2011 8:15 AM To: erlang-questions@REDACTED Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python Hi all, In my quest to learn Erlang I've translated Peter Norvig's sudoku solver into Erlang: https://github.com/apauley/sudoku-in-erlang For comparison, my slightly modified version of Norvig's Python code can be found here: https://github.com/apauley/sudoku-by-norvig I like the pattern matching, it was fun to do the entire solution without a single if or case statement :-) Something that bothers me though is that the Python solution seems to be faster, even after I've made the Erlang solution use multiple processors. In order to compare the two solutions I counted the number of eliminations each performed. The eliminate function is the core of the solution, for example assigning a single value to a square is implemented as the elimination of all other values. With the Erlang implementation I get: sudoku-in-erlang$ ./sudoku All tests passed :-) Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). And with the Python implementation: sudoku-by-norvig$ ./sudoku.py All tests pass. Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) (33059 total eliminations, avg 661.00, median 648, max 830, min 648). Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). So according to the stats above, the Python solution performs less computations when given exactly the same input. The Erlang code is as close to the Python as I could make it, I've done more or less a direct translation of the algorithms used. I suspect that there are some lazy evaluation happening in the Python version, possibly generators, although I haven't pinpointed it yet. How can I improve my Erlang code in this solution? Kind regards, Andreas _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From hd2010@REDACTED Sat Mar 26 06:29:02 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 26 Mar 2011 06:29:02 +0100 Subject: [erlang-questions 19] Re: fun with the new erlang.org! In-Reply-To: <20110325185008.GA32567@ecn.lan> References: <20110325185008.GA32567@ecn.lan> Message-ID: <4D8D799E.7020603@eonblast.com> On 3/25/11 7:50 PM, Michael Santos wrote: > And the WINNAR! Wow, congrats! I'll totally miss the old one, you get used to things and it was Erlang's face during an exciting time. But this is beautiful and inspiring, thanks a lot to the makers! Henning -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/00d07271/attachment.htm> From apauley@REDACTED Sat Mar 26 09:41:03 2011 From: apauley@REDACTED (Andreas Pauley) Date: Sat, 26 Mar 2011 10:41:03 +0200 Subject: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> Message-ID: <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> Thanks, I've changed the compile options and this definitely makes it somewhat faster: https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb Strangely the native flag is not documented here: http://www.erlang.org/doc/man/compile.html#file-2 The more interesting improvement would be to decrease the number of eliminations performed, but for that I'll have to go deep into the code :-) On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> wrote: > Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): > > Without native set: > 18> c(sudoku). > {ok,sudoku} > 19> ?sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > ok > > > With native set: > 20> c(sudoku,[native]). > {ok,sudoku} > 21> ?sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas Pauley > Sent: Friday, March 25, 2011 8:15 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python > > Hi all, > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > solver into Erlang: > https://github.com/apauley/sudoku-in-erlang > > For comparison, my slightly modified version of Norvig's Python code > can be found here: > https://github.com/apauley/sudoku-by-norvig > > I like the pattern matching, it was fun to do the entire solution > without a single if or case statement :-) > Something that bothers me though is that the Python solution seems to > be faster, even after I've made the Erlang solution use multiple > processors. > > In order to compare the two solutions I counted the number of > eliminations each performed. > The eliminate function is the core of the solution, for example > assigning a single value to a square is implemented as the elimination > of all other values. > > With the Erlang implementation I get: > sudoku-in-erlang$ ./sudoku > All tests passed :-) > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). > > And with the Python implementation: > sudoku-by-norvig$ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > ?(33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > So according to the stats above, the Python solution performs less > computations when given exactly the same input. > The Erlang code is as close to the Python as I could make it, I've > done more or less a direct translation of the algorithms used. > > I suspect that there are some lazy evaluation happening in the Python > version, possibly generators, although I haven't pinpointed it yet. > > How can I improve my Erlang code in this solution? > > Kind regards, > Andreas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Sat Mar 26 11:39:32 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 26 Mar 2011 11:39:32 +0100 Subject: [erlang-questions 21] Re: dialyzer and ssl In-Reply-To: <AANLkTimkauACTQJaRjKKktAGoJZGOyPWsJiFTLBbQPCn@mail.gmail.com> References: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> <AANLkTimkauACTQJaRjKKktAGoJZGOyPWsJiFTLBbQPCn@mail.gmail.com> Message-ID: <AANLkTinAP33MTFOSqkEYFyV2NxfStvrdyHQLayLv0Wn6@mail.gmail.com> hi anders, transport_accept/2 does return {ok, NewSocket} | {error, Reason} in my example here below. plus, it's the ssl_accept/2 that dialyzer is complaining on, at line 4. 2011/3/25 Anders Nygren <anders.nygren@REDACTED> > According to the documentation transport_accept/2 returns > {ok, NewSocket} | {error, Reason} > Which is what dialyzer also claims. So Your case clause on line 4 > can never match. > > /Anders > > On Thu, Mar 24, 2011 at 2:27 PM, Roberto Ostinelli <roberto@REDACTED> > wrote: > > dear list, > > > > i'm getting this dialyzer warning i cannot understand: > > > > "The pattern 'ok' can never match the type {'error',_} | > > {'ok',{'sslsocket',_,_}}" > > > > this points to line 4 of this code extract: > > > > 1. case ssl:transport_accept(ListenSocket) of > > 2. {ok, {sslsocket, _, _} = Sock} -> > > 3. case ssl:ssl_accept(Sock, 60000) of > > 4. ok -> > > ... > > 5. {error, _Reason} -> > > ... > > 6. end > > 7. {error, _Reason} -> > > ... > > 8. end > > > > afaik, ssl_accept/2 returns ok | {error, Reason} as per the specs: > > http://www.erlang.org/doc/man/ssl.html#ssl_accept-2 > > > > is this due to some discrepancy in docs/specs? or am i missing something? > > > > thank you, > > > > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/e0992923/attachment.htm> From alain.odea@REDACTED Sat Mar 26 13:26:39 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 26 Mar 2011 09:56:39 -0230 Subject: [erlang-questions 22] Re: [erlang-questions] Webmachine questions In-Reply-To: <AANLkTinxzOCdojfVCsHLpiJFVRwZb6rJmA1HX+nvozvf@mail.gmail.com> References: <3572825e-7f50-4f54-ad0c-337dada29c79@r13g2000yqk.googlegroups.com> <5206FCBE-8FF4-464F-8464-31C468926C2E@gmail.com> <AANLkTin2iShmS0gDqWHCCUneO5sLudfMA5+GM0H4TDBX@mail.gmail.com> <AANLkTi=zOFb5vfNYuWONdnD5-Uy9RzwCku9pykwcrZdx@mail.gmail.com> <AANLkTinxzOCdojfVCsHLpiJFVRwZb6rJmA1HX+nvozvf@mail.gmail.com> Message-ID: <2306B6D5-6BA2-4608-92C5-74FA08EAFD4E@gmail.com> Hi Ulf: Modifying the source of a framework to achieve your goals has serious drawbacks. It can introduce subtle bugs and can make updating the framework to follow security or performance fixes much harder. More importantly there is no advantage in modifying webmachine in this way since the behaviour you are looking for is cleanly achievable in resource code. I strongly advise you to reconsider altering the framework source. Having made that mistake myself and having experienced the maintenance nightmare it creates in a real product I want to save you the pain of experiencing it yourself :) Either way, plot out the pros and cons and clearly document the rationale for whatever decision you make. Best regards, Alain On 2011-03-26, at 2:24, Ulf Angermann <uangermann@REDACTED> wrote: > Hi Alain, > > you are right that i have my delete logic in seperate handler, but nevertheless it will POST and not a DELETE. > > I think about a solution, that i enhance the webmachine_mochiweb modul, so it will do the work for me. > In this modul i will ask when it is a post and there is a method=delete tunneld i will create a DELETE method. We'll see if this will work for me. > > XHR isn't a solution for me, because i want to use plain HTML without any JS, As i understood, you can't use it without. > > I will post my question to the webmachine mailing list, too. > > Thanks, > Ulf > > > 2011/3/25 Alain O'Dea <alain.odea@REDACTED> > Hi Ulf: > > You won't actually need two delete functions, since process_post can > delegate to your delete handler if the correct form parameter is > present. > > It's a compromise. Modifying the webmachine_decision_core to handle > protocol tunneling would be a serious challenge. An alternative would > be to have to have an HTTP proxy in front of webmachine that > understands your representation of tunneling. > > Overall a proxy would be epically more complicated and operational > problematic than a small amount of branching within a resource. A > much better alternative is to implement a generic webmachine resource > that accepts a callback module whose functions perform the > application-level request processing. That way you could effectively > have a generic HTTP method tunneling webmachine resource :) > > None of this is necessary if you use XHR to send your forms since it > allows you to set the method. > > Cheers, > Alain > > On Fri, Mar 25, 2011 at 1:58 AM, Ulf Angermann > <uangermann@REDACTED> wrote: > > Hi Alan, > > > > thanks for answer, but i am not really happy with your first one, because > > than i have to implement two "delete" functions. In my opinion it would be > > nicer, if i could change the method from POST to DELETE. Than i could handle > > the tunneld delete as the "normal" one. > > > > I will look at zotonic if i find something there, thank you. > > > > Thanks, > > Ulf > > > > 2011/3/24 Alain O'Dea <alain.odea@REDACTED> > >> > >> Hi Ulf: > >> > >> On 2011-03-24, at 5:45, u a <uaforum1@REDACTED> wrote: > >> > >> > Hello, > >> > > >> > i am working on a small project, where i am using webmachine. > >> > Everything works fine, but know i have some question regarding the > >> > tool. > >> > > >> > 1. Tunneld delete throw POST (HTML forms) > >> > > >> > Because of the limitation of the browser, i have to tunnel a delete > >> > request throw a post request to webmachine. > >> > When i look in the trace, great tool :-) , my request comes to the > >> > decision M16 where the question is asked if this is a DELETE method or > >> > not. > >> > The answer is false , because it is a POST and not a DELETE method. > >> > > >> > Is it possible to change the method from a POST to a DELETE or which > >> > kind of solution is the best? > >> > > >> > 2. Getting the protocol for the location > >> > > >> > When i create a location header or a redirect, i want to decide of the > >> > underlying socket, which kind of protocol i have to use (HTTP or HTTPS > >> > for instance. I found that the socket is part of webmachine_request, > >> > but there is no function inside wrq to retrieve this information. > >> > > >> > Is there an easy way to detect the protocol which is used inside the > >> > resource? > >> > > >> > Thanks for your help, > >> > Ulf > >> > > >> > >> 1. Your resource's process_post function needs to handle this. Since you > >> are tunneling DELETE through POST it is actually POST that you need to > >> handle. I am guessing your tunneling is based on a form param called > >> method. Read that and branch in process_post :) > >> > >> 2. No idea off the top of my head how to get the protocol. I don't think > >> vanilla webmachine wrq has this info. I'm pretty sure it is visible in > >> Zotonic webmachine resources though > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/d42842eb/attachment.htm> From kostis@REDACTED Sat Mar 26 13:45:35 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 26 Mar 2011 14:45:35 +0200 Subject: [erlang-questions 23] Re: dialyzer and ssl In-Reply-To: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> References: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> Message-ID: <4D8DDFEF.7080307@cs.ntua.gr> Roberto Ostinelli wrote: > dear list, > > i'm getting this dialyzer warning i cannot understand: > > "The pattern 'ok' can never match the type {'error',_} | > {'ok',{'sslsocket',_,_}}" > > this points to line 4 of this code extract: > > 1. case ssl:transport_accept(ListenSocket) of > 2. {ok, {sslsocket, _, _} = Sock} -> > 3. case ssl:ssl_accept(Sock, 60000) of > 4. ok -> > ... > 5. {error, _Reason} -> > ... > 6. end > 7. {error, _Reason} -> > ... > 8. end > > afaik, ssl_accept/2 returns ok | {error, Reason} as per the specs: > http://www.erlang.org/doc/man/ssl.html#ssl_accept-2 > > is this due to some discrepancy in docs/specs? or am i missing something? You have not told us which Erlang/OTP version this is, so it's hard to tell you what's happening. The documentation you are pointing us at is for R14B02. Is your code also using the dialyzer of this version? In R14B02 the spec for this function (from the ssl.erl code) reads: -spec ssl_accept(#sslsocket{} | port(), timeout()| [option()]) -> ok | {ok, #sslsocket{}} | {error, reason()}. which makes me strongly suspect that this function has changed return values between Erlang/OTP versions (it was previously returning 'ok' and now it returns {'ok', #sslsocket{}}, or the other way around). My advice: if you are to be relying on the Erlang/OTP documentation on the web, use R14B02. Kostis From roberto@REDACTED Sat Mar 26 14:21:24 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 26 Mar 2011 14:21:24 +0100 Subject: [erlang-questions 24] Re: dialyzer and ssl In-Reply-To: <4D8DDFEF.7080307@cs.ntua.gr> References: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> <4D8DDFEF.7080307@cs.ntua.gr> Message-ID: <AANLkTi=4j4tLp8hPPG4FbUgHNxD64R6bCJXG9v3DV1G_@mail.gmail.com> 2011/3/26 Kostis Sagonas <kostis@REDACTED> > > You have not told us which Erlang/OTP version this is, so it's hard to tell > you what's happening. The documentation you are pointing us at is for > R14B02. Is your code also using the dialyzer of this version? > > In R14B02 the spec for this function (from the ssl.erl code) reads: > > -spec ssl_accept(#sslsocket{} | port(), timeout()| [option()]) -> > ok | {ok, #sslsocket{}} | {error, reason()}. > > which makes me strongly suspect that this function has changed return > values between Erlang/OTP versions (it was previously returning 'ok' and now > it returns {'ok', #sslsocket{}}, or the other way around). > > My advice: if you are to be relying on the Erlang/OTP documentation on the > web, use R14B02. > > Kostis > hi kostis, i'm using R13B04, and i've built the .dialyzer_plt file against this version. to improve the compatibility with previous versions, i've therefore added: case ssl:ssl_accept(Sock, 60000) of ok -> ...; {ok, NewSock} -> ...; {error, _Reason} -> ... end dialyzer is still complaining obviously since it is not waiting for an 'ok' match, however i know by empirical tests that it's the 'ok' being matched, not the '{ok, NewSock}', in R13B04. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/e68ca327/attachment.htm> From ulf.wiger@REDACTED Sat Mar 26 15:35:18 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 26 Mar 2011 15:35:18 +0100 Subject: [erlang-questions 25] edown update 2 - generates top-level README In-Reply-To: <AF962161-B14D-44FD-9E58-9FA033AFA59E@erlang-solutions.com> References: <AF962161-B14D-44FD-9E58-9FA033AFA59E@erlang-solutions.com> Message-ID: <F384122F-F318-4103-A102-A399B3E5B506@erlang-solutions.com> My last post apparently triggered some people to start using edown, or at least contact me with bug reports. Thank you for that. :) I have updated edown to include R14B02 additions, and corrected some layout bugs (thanks Steve V. for your patience). I have now added a edown-specific option, 'top_level_readme', which rewrites the hyperlinks in doc/README.md and puts the result in the top-level directory. This way, you can maintain the overview.edoc file and have edown automatically produce a README.md for your github repos. Details at https://github.com/esl/edown BR, Ulf W PS As suggested by Paolo Negri (@hungryblank), I have also started tagging some of the commits of edown and gproc (will do so for other repos as well, in good time). The latest edown tag is 'v0.2.0'. On 23 Mar 2011, at 08:49, Ulf Wiger wrote: > > I recently pushed a nice little contribution to edown, called markedoc. > > http://github.com/esl/edown > > Edown renders Edoc markup as Markdown, in order to make the docs easy > to read while browsing a Github repository. > > The easiest way to use it is with rebar, > > {deps, [{edown, ".*", {git, "git://github.com/esl/edown.git", "HEAD"}}]}. > {edoc_opts, [{doclet, edown_doclet}, > {src_path, ["src/"]}, > {subpackages, true}]}. > > then ./rebar doc will generate .md files, including a doc/README.md > Copying doc/README.md to the parent directory will break links in the module > index. These can be manually fixed, or you can try to automate it using something > like https://github.com/esl/run_eqc/blob/master/mk_readme.escript > (I'm working on making that more generic, but that seems to be quite a step > up in complexity). > > MarkEdoc goes the other way, converting Markdown to Edoc markup. > The main use would be to maintain the README.md file as regular Markdown, > and convert it to a proper overview.edoc using MarkEdoc. > > https://github.com/esl/edown/blob/master/bin/MARKEDOC-README.md > > Many thanks to Henning Dietrich for this contribution. > > BR, > Ulf W > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/ce1b40b6/attachment.htm> From jwilberding@REDACTED Sat Mar 26 16:22:46 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Sat, 26 Mar 2011 11:22:46 -0400 Subject: [erlang-questions 26] Ignoring bad beams with dialyzer Message-ID: <AANLkTik5QKCr1an3mzr24CKRNukEhVYnRcvrHg8Uap-m@mail.gmail.com> Is there a way to make dialyzer ignore beams that weren't compiled with +debug_info instead of erroring out? Is there a reason it has to error out? Thanks! Jordan Wilberding -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/51862ff8/attachment.htm> From kostis@REDACTED Sat Mar 26 16:23:48 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 26 Mar 2011 17:23:48 +0200 Subject: [erlang-questions 27] Re: Ignoring bad beams with dialyzer In-Reply-To: <AANLkTik5QKCr1an3mzr24CKRNukEhVYnRcvrHg8Uap-m@mail.gmail.com> References: <AANLkTik5QKCr1an3mzr24CKRNukEhVYnRcvrHg8Uap-m@mail.gmail.com> Message-ID: <4D8E0504.4010104@cs.ntua.gr> Jordan Wilberding wrote: > Is there a way to make dialyzer ignore beams that weren't compiled with > +debug_info instead of erroring out? Yes: provide a patch for it ;-) (Preferably with a new command line option.) > Is there a reason it has to error out? Well, you presumably asked dialyzer to dialyze a bunch of beam files and it can only handle some of them. No wonder it feels very sad about it! Cheers, Kostis From kostis@REDACTED Sat Mar 26 16:37:18 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 26 Mar 2011 17:37:18 +0200 Subject: [erlang-questions 28] Re: Ignoring bad beams with dialyzer In-Reply-To: <4D8E0504.4010104@cs.ntua.gr> References: <AANLkTik5QKCr1an3mzr24CKRNukEhVYnRcvrHg8Uap-m@mail.gmail.com> <4D8E0504.4010104@cs.ntua.gr> Message-ID: <4D8E082E.2010509@cs.ntua.gr> Kostis Sagonas wrote: > Jordan Wilberding wrote: >> Is there a way to make dialyzer ignore beams that weren't compiled >> with +debug_info instead of erroring out? > > Yes: provide a patch for it ;-) > (Preferably with a new command line option.) > >> Is there a reason it has to error out? > > Well, you presumably asked dialyzer to dialyze a bunch of beam files and > it can only handle some of them. No wonder it feels very sad about it! Just in case it was not obvious... I was half-joking in my reply. This is a very good suggestion! If noone beats us to it and provide a patch for this, we will implement it for the next OTP release. Kostis From antoine.koener@REDACTED Sat Mar 26 17:03:56 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Sat, 26 Mar 2011 17:03:56 +0100 Subject: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <4D8D799E.7020603@eonblast.com> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> Message-ID: <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> Cacheability ? I can reload the page multiples times and still download images. It seems also that images don't have any 'cache' attributes. (I don't recommended using ETags unless you're absolutely sure that ETags will be always the same on multiple web servers. i.e. some webservers use inode information to compute the ETag which is different on different servers...thought I don't know how erlang-web compute it) Furthermore all links in the 'Articles' part seems dead (http://www.erlang.org/doc.html ) From jwilberding@REDACTED Sat Mar 26 19:51:20 2011 From: jwilberding@REDACTED (Jordan Wilberding) Date: Sat, 26 Mar 2011 14:51:20 -0400 Subject: [erlang-questions 30] Re: Ignoring bad beams with dialyzer In-Reply-To: <4D8E082E.2010509@cs.ntua.gr> References: <AANLkTik5QKCr1an3mzr24CKRNukEhVYnRcvrHg8Uap-m@mail.gmail.com> <4D8E0504.4010104@cs.ntua.gr> <4D8E082E.2010509@cs.ntua.gr> Message-ID: <AANLkTi=oFhJP7t+oUCu1gzC7H-ja3xsjmTLQtzpzXxti@mail.gmail.com> I am more than happy to provide a patch. Be expecting one soon. Thanks! Jordan Wilberding On Sat, Mar 26, 2011 at 11:37 AM, Kostis Sagonas <kostis@REDACTED> wrote: > Kostis Sagonas wrote: > >> Jordan Wilberding wrote: >> >>> Is there a way to make dialyzer ignore beams that weren't compiled with >>> +debug_info instead of erroring out? >>> >> >> Yes: provide a patch for it ;-) >> (Preferably with a new command line option.) >> >> Is there a reason it has to error out? >>> >> >> Well, you presumably asked dialyzer to dialyze a bunch of beam files and >> it can only handle some of them. No wonder it feels very sad about it! >> > > Just in case it was not obvious... I was half-joking in my reply. > > This is a very good suggestion! If noone beats us to it and provide a > patch for this, we will implement it for the next OTP release. > > Kostis > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/ad553b87/attachment.htm> From spawn.think@REDACTED Sat Mar 26 20:08:01 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sat, 26 Mar 2011 20:08:01 +0100 Subject: [erlang-questions 31] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> Message-ID: <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> > wrote: > > Without going deep into the code, one thing to try is compile with the > native flag (running the tests from the VM shell here): > > > > Without native set: > > 18> c(sudoku). > > {ok,sudoku} > > 19> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > ok > > > > > > With native set: > > 20> c(sudoku,[native]). > > {ok,sudoku} > > 21> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > > > > > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] On Behalf Of Andreas Pauley > > Sent: Friday, March 25, 2011 8:15 AM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > Python > > > > Hi all, > > > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > > solver into Erlang: > > https://github.com/apauley/sudoku-in-erlang > > > > For comparison, my slightly modified version of Norvig's Python code > > can be found here: > > https://github.com/apauley/sudoku-by-norvig > > > > I like the pattern matching, it was fun to do the entire solution > > without a single if or case statement :-) > > Something that bothers me though is that the Python solution seems to > > be faster, even after I've made the Erlang solution use multiple > > processors. > > > > In order to compare the two solutions I counted the number of > > eliminations each performed. > > The eliminate function is the core of the solution, for example > > assigning a single value to a square is implemented as the elimination > > of all other values. > > > > With the Erlang implementation I get: > > sudoku-in-erlang$ ./sudoku > > All tests passed :-) > > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > 1770). > > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > 1781). > > > > And with the Python implementation: > > sudoku-by-norvig$ ./sudoku.py > > All tests pass. > > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > > > So according to the stats above, the Python solution performs less > > computations when given exactly the same input. > > The Erlang code is as close to the Python as I could make it, I've > > done more or less a direct translation of the algorithms used. > > > > I suspect that there are some lazy evaluation happening in the Python > > version, possibly generators, although I haven't pinpointed it yet. > > > > How can I improve my Erlang code in this solution? > > > > Kind regards, > > Andreas > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/a2ee66a9/attachment.htm> From mevans@REDACTED Sat Mar 26 20:46:20 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 26 Mar 2011 15:46:20 -0400 Subject: [erlang-questions 32] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com>, <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB26@vvexch.verivue.com> Cool... Might also want to look at what happens if all the other Erlang modules (esp. lists, sets and dict for your app) are made native: Without native set: 18> c(sudoku). {ok,sudoku} 19> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok With native set: 20> c(sudoku,[native]). {ok,sudoku} > sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok Dict/lists/sets native too: 5> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.996860 secs (95.30 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok Over a 2x boost in performance with zero re-factoring of your original code....maybe I'll put a question to the group out there. Seems that they should be shouting this sort of improvement from the roof-tops... Matt ________________________________________ From: Andreas Pauley [apauley@REDACTED] Sent: Saturday, March 26, 2011 4:41 AM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions 4] A sudoku solver in Erlang compared to Python Thanks, I've changed the compile options and this definitely makes it somewhat faster: https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb Strangely the native flag is not documented here: http://www.erlang.org/doc/man/compile.html#file-2 The more interesting improvement would be to decrease the number of eliminations performed, but for that I'll have to go deep into the code :-) On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> wrote: > Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): > > Without native set: > 18> c(sudoku). > {ok,sudoku} > 19> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > ok > > > With native set: > 20> c(sudoku,[native]). > {ok,sudoku} > 21> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas Pauley > Sent: Friday, March 25, 2011 8:15 AM > To: erlang-questions@REDACTED > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python > > Hi all, > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > solver into Erlang: > https://github.com/apauley/sudoku-in-erlang > > For comparison, my slightly modified version of Norvig's Python code > can be found here: > https://github.com/apauley/sudoku-by-norvig > > I like the pattern matching, it was fun to do the entire solution > without a single if or case statement :-) > Something that bothers me though is that the Python solution seems to > be faster, even after I've made the Erlang solution use multiple > processors. > > In order to compare the two solutions I counted the number of > eliminations each performed. > The eliminate function is the core of the solution, for example > assigning a single value to a square is implemented as the elimination > of all other values. > > With the Erlang implementation I get: > sudoku-in-erlang$ ./sudoku > All tests passed :-) > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). > > And with the Python implementation: > sudoku-by-norvig$ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > So according to the stats above, the Python solution performs less > computations when given exactly the same input. > The Erlang code is as close to the Python as I could make it, I've > done more or less a direct translation of the algorithms used. > > I suspect that there are some lazy evaluation happening in the Python > version, possibly generators, although I haven't pinpointed it yet. > > How can I improve my Erlang code in this solution? > > Kind regards, > Andreas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mevans@REDACTED Sat Mar 26 20:56:12 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 26 Mar 2011 15:56:12 -0400 Subject: [erlang-questions 33] HiPE or native....good results, but confused... Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> Hi, If anyone has been following the Sudoku email thread started by Andreas, I suggested he looked at compiling his code with the "native" flag set. With his app, and some common Erlang modules compiled to be native (espc. lists, sets and dict) I got a 2x performance improvement (see below). I hadn't really played with HiPE (native) much, as I was always under the impression that performance gains were negligible. Clearly, for this use case, the gains are far from negligible - a 2x boost is not something to ignore. My questions, simply put are: 1) When should HiPE be used? 2) When shouldn't HiPE be used? 3) What are the disadvantages of using HiPE? 4) Shouldn't "team Erlang" be doing more to promote its use? 5) Are any HiPE changes on the road-map? Thanks Matt Results..... Without native set: 18> c(sudoku). {ok,sudoku} 19> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok With native set: 20> c(sudoku,[native]). {ok,sudoku} > sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok Dict/lists/sets native too: 5> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.996860 secs (95.30 Hz) (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). ok From bob@REDACTED Sat Mar 26 20:57:41 2011 From: bob@REDACTED (Bob Cowdery) Date: Sat, 26 Mar 2011 19:57:41 +0000 Subject: [erlang-questions 34] Re: [erlang-questions] Binary no-copy In-Reply-To: <AANLkTinruhUL_BV04m0Qti0eAavp1VEFkU75SFVP+qRh@mail.gmail.com> References: <4D82315F.5030201@bobcowdery.plus.com> <AANLkTi=6ysXnP_S2xtfcJNWQo_WtFeN2xWKg9VbTF6mL@mail.gmail.com> <4D825391.5040201@bobcowdery.plus.com> <AANLkTim-U2=D9Sa49VtyteBgUV5AU2iRhBfXcR8LHd+w@mail.gmail.com> <4D8361D3.3060202@bobcowdery.plus.com> <AANLkTinruhUL_BV04m0Qti0eAavp1VEFkU75SFVP+qRh@mail.gmail.com> Message-ID: <4D8E4535.7060609@bobcowdery.plus.com> Paul Thank you for the explanations. I have data running right through the app now using a simulator in Erlang pretending to be the hardware. Next up is to plug in the DSP. I've done a linked-in driver before so hoping that won't be too difficult. Bob On 23/03/2011 01:50, Per Gustafsson wrote: > So essentially everytime this function is called one of the four > clauses will match > > 1. Will match if the state is exactly a full block right now. It will > send that frame to the dsp module, and then it will continue to decode > the current frame using a new state. > > 2. If the current state is larger than a full block we will split out > the full block and send that to the dsp module, we will then continue > to decode the current frame using whatever was left in the old state > as the new state. > > 3. If the frame still has samples in it and neither of the first two > cases were true we will take out one set of samples, convert to them > to floats and add them to the end of the state and continue to decode > the rest of the frame. > > 4 If the frame is empty we will return the current state. > > The difference between this version and the version that uses a > comprehension is that we are checking for every sample whether the > state is now a full frame or not. > > This allows us to write the new samples directly into the state > binary, rather than creating a temporary binary. That we then copy > into the state binary. > > > 2011/3/18 Bob Cowdery <bob@REDACTED > <mailto:bob@REDACTED>> > > Humour me. I'm still learning erlang and it still takes me ages to > write a few lines at times. Comments in-line. > > Bob > > > On 17/03/2011 20:54, Per Gustafsson wrote: >> You're right it should be as below to get the same behaviour, but >> I think it will work the same for input that makes sense. >> > Straight forward > >> do_decode_frame(Frame, State) -> >> %% Separate the command bytes from the frame data >> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame, >> decode_samples(Rest, State). >> > First clause will match a full block where frames and blocks are > aligned. Never happen in practice as 63 will never be a multiple > of 64 but it's right that the code should not know this. > >> decode_samples(Frame, FullBlock) when bytesize(FullBlock) == >> ?BinSize -> >> dsp:process_block(FullBlock), >> decode_samples(Frame, <<>>); > Second clause will match when a FullBlock has > ?BinSize and will > split into ?BinSize and Rest >> decode_samples(Frame, <<FullBlock:?BinSize/binary, Rest/binary>>) -> >> dsp:process_block(FullBlock), >> decode_samples(Frame, Rest); > Third clause. Now I get lost. This will match pretty much any > binary > 8 bytes and any State. So this will get called from first > and second clause. The next two clauses seem to run into each > other. Don't understand that. Is the binary comprehension hidden > somewhere in there? > > > Sorry the third clause is totally wrong. It should have been: > decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed, > Rest/binary>>, State) -> > decode_samples(Rest, <<State/binary, (I/?S):32/float, > (Q/?S):32/float>>); > >> decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed, >> Rest/binary>>, State) -> >> decode_samples(<<State/binary, (I/?S):32/float, (Q/?S):32/float>>; >> decode_samples(<<>>, State) -> >> {ok, State}. >> > Would this work as the last two clauses: > decode_samples(Frame, State) -> > decode_samples(<<>>, << State/binary, <<(I/?S):32/float, > (Q/?S):32/float>> || <<I:24/big-signed, Q:24/big-signed, > _M:16/big-signed>> <= Frame >>) > > decode_samples(<<>>, State) -> > {ok, State}. >> >> 2011/3/17 Bob Cowdery <bob@REDACTED >> <mailto:bob@REDACTED>>: >> > Thanks. More or less what I thought that the only copy is in >> the append >> > operation. The one liner is the big one in terms of time but I will >> > profile the whole thing eventually. I don't think your solution >> does >> > quite the same thing as the sent block must be exactly the >> correct size >> > not >= but I appreciate the kind of structure you are using >> with small >> > clauses is more elegant. >> > >> > Bob >> > >> > On 17/03/2011 17:11, Per Gustafsson wrote: >> >> It will copy binaries, but essentially the state binary will >> act like >> >> a growable array, pre-allocating space to grow in, so there >> will be >> >> some copying , but it will be linear in the size of the resulting >> >> array. >> >> It might be a little bit faster to rewrite this as shown >> below. The >> >> reason I think this might be faster is that you do not create a >> >> temporary samples array that just gets copied. Whether this is >> faster >> >> or not in reality is of course a matter of benchmarking. >> >> >> >> %% Constants >> >> %% Input scale factor >> >> -define (S, (1 bsl 23) - 1). >> >> %% Block size for DSP (interleaved I and Q) >> >> -define (BlockSz, 1024). >> >> %% Samples in a Metis frame (interleaved I and Q) >> >> -define (MetisSmpls, 63). >> >> %% Bytes in a float >> >> -define (FloatSz, 4). >> >> %% Size of binary for given block size >> >> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). >> >> >> >> do_decode_frame(Frame, State) -> >> >> %% Separate the command bytes from the frame data >> >> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame, >> >> decode_samples(Rest, State). >> >> >> >> decode_samples(Frame, State) when size(State) >= ?BinSz -> >> >> dsp:process_block(State), >> >> decode_samples(Frame, <<>>); >> >> decode_samples(<<I:24/big-signed, Q:24/big-signed, >> _M:16/big-signed, >> >> Rest/binary>>, State) -> >> >> decode_samples(<<State/binary, (I/?S):32/float, >> (Q/?S):32/float>>; >> >> decode_samples(<<>>, State) -> >> >> {ok, State}. >> >> >> >> 2011/3/17 Bob Cowdery <bob@REDACTED >> <mailto:bob@REDACTED>> >> >>> Hi, >> >>> >> >>> Am I correct in thinking that the code fragments below will >> not copy any >> >>> binary data. I've not tested the function yet so it may have >> bugs but >> >>> I'm interested in whether it will be optimal in efficiency. >> State starts >> >>> off as an empty binary. Frames are decoded and placed in a >> block. There >> >>> are not an integral number of frames to a block. The function >> >>> dsp:process_block() is in another gen-server. >> >>> >> >>> Thanks >> >>> Bob >> >>> >> >>> %% Constants >> >>> %% Input scale factor >> >>> -define (S, (1 bsl 23) - 1). >> >>> %% Block size for DSP (interleaved I and Q) >> >>> -define (BlockSz, 1024). >> >>> %% Samples in a Metis frame (interleaved I and Q) >> >>> -define (MetisSmpls, 63). >> >>> %% Bytes in a float >> >>> -define (FloatSz, 4). >> >>> %% Size of binary for given block size >> >>> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls). >> >>> >> >>> do_decode_frame(Frame, State) -> >> >>> %% Separate the command bytes from the frame data >> >>> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame, >> >>> %% What's left now is <<I:24, Q:24, M:16 ..repeated 63 >> times.. >> >> >>> %% This one liner converts the I and Q to scaled floats >> >>> Samples = << <<(I/?S):32/float, (Q/?S):32/float>> || >> >>> <<I:24/big-signed, Q:24/big-signed, _M:16/big-signed>> <= >> Rest >>, >> >>> CurrentSz = size(State), >> >>> FrameSz = size(Samples), >> >>> if >> >>> (CurrentSz + FrameSz) < ?BinSz -> >> >>> NewState = <<State/binary, Samples/binary>>; >> >>> true -> >> >>> %% Binary part of this transfer >> >>> CutAt = ?BinSz - CurrentSz - 1, >> >>> Bp = binary_part(Samples, {0, CutAt}), >> >>> %% Binary part of remainder >> >>> Br = binary_part(Samples, {CutAt, FrameSz - CutAt}), >> >>> %% Process this block >> >>> dsp:process_block(<<State/binary, Bp/binary>>), >> >>> NewState = Br >> >>> end, >> >>> {ok, NewState}. >> >>> >> >>> ________________________________________________________________ >> >>> erlang-questions (at) erlang.org <http://erlang.org> mailing >> list. >> >>> See http://www.erlang.org/faq.html >> >>> To unsubscribe; >> mailto:erlang-questions-unsubscribe@REDACTED >> <mailto:erlang-questions-unsubscribe@REDACTED> >> >>> >> > >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/f4a38a98/attachment.htm> From mevans@REDACTED Sat Mar 26 21:14:56 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 26 Mar 2011 16:14:56 -0400 Subject: [erlang-questions 35] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com>, <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> Good call....gb_sets should be faster. Compiled to native it runs faster still 9> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). ok ________________________________________ From: Ahmed Omar [spawn.think@REDACTED] Sent: Saturday, March 26, 2011 3:08 PM To: Andreas Pauley Cc: Evans, Matthew; erlang-questions@REDACTED Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: Thanks, I've changed the compile options and this definitely makes it somewhat faster: https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb Strangely the native flag is not documented here: http://www.erlang.org/doc/man/compile.html#file-2 The more interesting improvement would be to decrease the number of eliminations performed, but for that I'll have to go deep into the code :-) On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): > > Without native set: > 18> c(sudoku). > {ok,sudoku} > 19> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > ok > > > With native set: > 20> c(sudoku,[native]). > {ok,sudoku} > 21> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED> [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>] On Behalf Of Andreas Pauley > Sent: Friday, March 25, 2011 8:15 AM > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python > > Hi all, > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > solver into Erlang: > https://github.com/apauley/sudoku-in-erlang > > For comparison, my slightly modified version of Norvig's Python code > can be found here: > https://github.com/apauley/sudoku-by-norvig > > I like the pattern matching, it was fun to do the entire solution > without a single if or case statement :-) > Something that bothers me though is that the Python solution seems to > be faster, even after I've made the Erlang solution use multiple > processors. > > In order to compare the two solutions I counted the number of > eliminations each performed. > The eliminate function is the core of the solution, for example > assigning a single value to a square is implemented as the elimination > of all other values. > > With the Erlang implementation I get: > sudoku-in-erlang$ ./sudoku > All tests passed :-) > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). > > And with the Python implementation: > sudoku-by-norvig$ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > So according to the stats above, the Python solution performs less > computations when given exactly the same input. > The Erlang code is as close to the Python as I could make it, I've > done more or less a direct translation of the algorithms used. > > I suspect that there are some lazy evaluation happening in the Python > version, possibly generators, although I haven't pinpointed it yet. > > How can I improve my Erlang code in this solution? > > Kind regards, > Andreas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> http://erlang.org/mailman/listinfo/erlang-questions -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think<http://twitter.com/#!/spawn_think> From kennethstone@REDACTED Sat Mar 26 21:21:22 2011 From: kennethstone@REDACTED (Kenny Stone) Date: Sat, 26 Mar 2011 15:21:22 -0500 Subject: [erlang-questions 36] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> Message-ID: <AANLkTinY5HuOS5cCjqxPUFtn0Zuhq0DnC7MWpxyEiAeP@mail.gmail.com> Are you solving the puzzles in parallel? It might be revealing to see the implementation differences in python and erlang for solving a lot of puzzles when the code parallelized. The erlang code won't change much. On Sat, Mar 26, 2011 at 3:14 PM, Evans, Matthew <mevans@REDACTED> wrote: > Good call....gb_sets should be faster. Compiled to native it runs faster > still > > 9> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED] > Sent: Saturday, March 26, 2011 3:08 PM > To: Andreas Pauley > Cc: Evans, Matthew; erlang-questions@REDACTED > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared > to Python > > using gb_sets instead of sets could decrease eliminations a bit and give u > some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto: > apauley@REDACTED>> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED > <mailto:mevans@REDACTED>> wrote: > > Without going deep into the code, one thing to try is compile with the > native flag (running the tests from the VM shell here): > > > > Without native set: > > 18> c(sudoku). > > {ok,sudoku} > > 19> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > ok > > > > > > With native set: > > 20> c(sudoku,[native]). > > {ok,sudoku} > > 21> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > > > > > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED> [mailto: > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>] On Behalf Of Andreas Pauley > > Sent: Friday, March 25, 2011 8:15 AM > > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > Python > > > > Hi all, > > > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > > solver into Erlang: > > https://github.com/apauley/sudoku-in-erlang > > > > For comparison, my slightly modified version of Norvig's Python code > > can be found here: > > https://github.com/apauley/sudoku-by-norvig > > > > I like the pattern matching, it was fun to do the entire solution > > without a single if or case statement :-) > > Something that bothers me though is that the Python solution seems to > > be faster, even after I've made the Erlang solution use multiple > > processors. > > > > In order to compare the two solutions I counted the number of > > eliminations each performed. > > The eliminate function is the core of the solution, for example > > assigning a single value to a square is implemented as the elimination > > of all other values. > > > > With the Erlang implementation I get: > > sudoku-in-erlang$ ./sudoku > > All tests passed :-) > > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > 1770). > > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > 1781). > > > > And with the Python implementation: > > sudoku-by-norvig$ ./sudoku.py > > All tests pass. > > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > > > So according to the stats above, the Python solution performs less > > computations when given exactly the same input. > > The Erlang code is as close to the Python as I could make it, I've > > done more or less a direct translation of the algorithms used. > > > > I suspect that there are some lazy evaluation happening in the Python > > version, possibly generators, although I haven't pinpointed it yet. > > > > How can I improve my Erlang code in this solution? > > > > Kind regards, > > Andreas > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/63aad8ee/attachment.htm> From spawn.think@REDACTED Sat Mar 26 21:33:40 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sat, 26 Mar 2011 21:33:40 +0100 Subject: [erlang-questions 37] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> Message-ID: <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> actually instead of doing NonUniquePeers = shallow_flatten([S || S <- units(Square)]), PeerSet = sets:from_list(NonUniquePeers), PeersWithSelf = sets:to_list(PeerSet), can't u just do PeersWithSelf = lists:usort(NonUniquePeers). ? On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED> wrote: > Good call....gb_sets should be faster. Compiled to native it runs faster > still > > 9> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED] > Sent: Saturday, March 26, 2011 3:08 PM > To: Andreas Pauley > Cc: Evans, Matthew; erlang-questions@REDACTED > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared > to Python > > using gb_sets instead of sets could decrease eliminations a bit and give u > some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto: > apauley@REDACTED>> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED > <mailto:mevans@REDACTED>> wrote: > > Without going deep into the code, one thing to try is compile with the > native flag (running the tests from the VM shell here): > > > > Without native set: > > 18> c(sudoku). > > {ok,sudoku} > > 19> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > ok > > > > > > With native set: > > 20> c(sudoku,[native]). > > {ok,sudoku} > > 21> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > > > > > > > -----Original Message----- > > From: erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED> [mailto: > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>] On Behalf Of Andreas Pauley > > Sent: Friday, March 25, 2011 8:15 AM > > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > Python > > > > Hi all, > > > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > > solver into Erlang: > > https://github.com/apauley/sudoku-in-erlang > > > > For comparison, my slightly modified version of Norvig's Python code > > can be found here: > > https://github.com/apauley/sudoku-by-norvig > > > > I like the pattern matching, it was fun to do the entire solution > > without a single if or case statement :-) > > Something that bothers me though is that the Python solution seems to > > be faster, even after I've made the Erlang solution use multiple > > processors. > > > > In order to compare the two solutions I counted the number of > > eliminations each performed. > > The eliminate function is the core of the solution, for example > > assigning a single value to a square is implemented as the elimination > > of all other values. > > > > With the Erlang implementation I get: > > sudoku-in-erlang$ ./sudoku > > All tests passed :-) > > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > 1770). > > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > 1781). > > > > And with the Python implementation: > > sudoku-by-norvig$ ./sudoku.py > > All tests pass. > > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > > > So according to the stats above, the Python solution performs less > > computations when given exactly the same input. > > The Erlang code is as close to the Python as I could make it, I've > > done more or less a direct translation of the algorithms used. > > > > I suspect that there are some lazy evaluation happening in the Python > > version, possibly generators, although I haven't pinpointed it yet. > > > > How can I improve my Erlang code in this solution? > > > > Kind regards, > > Andreas > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/405282b1/attachment.htm> From jesper.louis.andersen@REDACTED Sat Mar 26 21:34:07 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 26 Mar 2011 21:34:07 +0100 Subject: [erlang-questions 38] Re: HiPE or native....good results, but confused... In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> Message-ID: <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> On Sat, Mar 26, 2011 at 20:56, Evans, Matthew <mevans@REDACTED> wrote: > My questions, simply put are: > > 1) When should HiPE be used? When the problem is mostly CPU-bound, you can often gain a factor of 2-5 from using HiPE. Beware there is an overhead in switching to/from HiPE, so if you intermix modules which are HiPE'd and not, you will often incur a penalty in performance. So it is important to compile all used modules into native format. That includes stdlib, so beware of your calls to those. > 2) When shouldn't HiPE be used? For some problems, you are not waiting that much on the CPU, but more time is spent waiting on other subsystems, disk, the database, the network interface and so on. In those cases, the gains from switching to HiPE is neglicible. The programs I work on myself hardly benefits from enabling HiPE. We are talking very little CPU-time spared by doing it. (rant coming) In general, the main reason could be that Erlang is not really built for CPU-bound computation. With current compiler tech, the fastest languages are *clearly* the statically typed languages. Yes, V8+crankshaft is impressive, but it still cannot beat a well-laid-out Haskell or Ocaml program. Common Lisp can be fast, but you need to type the program yourself through type hints to the CL compiler (See CMUCL or SBCL for instance). You are far better off outsourcing the computationally heavy parts. There are at least 4 different ways to get that done in Erlang, (hidden node, port, port_driver, NIF) so you have plenty of options. Then again, people are too obsessed with performance in the first place. Often, it is far more important to think and choose the algorithms and datastructures so they are the right kinds. If you do this well, then your interpreted program can easily be as fast or faster than natively compiled counterparts. This is *especially* true for programs written in weak languages such as C and Java where people don't have access to abstraction features and hence bake everything in with the lowest common denominator. > With native set: > 20> c(sudoku,[native]). Try c(sudoku, [native, {hipe, o3}]) to give it a tad more optimization to work with. It is often faster. -- J. From spawn.think@REDACTED Sat Mar 26 21:39:13 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sat, 26 Mar 2011 21:39:13 +0100 Subject: [erlang-questions 39] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> Message-ID: <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> BTW, beside looking into reducing eleminations (which requires deep dive), if u do some time profiling using eprof u will find that most of the time spent actually in the cross functions sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 35024918 [ 0.36] On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > actually instead of doing > > NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > PeerSet = sets:from_list(NonUniquePeers), > PeersWithSelf = sets:to_list(PeerSet), > > can't u just do > PeersWithSelf = lists:usort(NonUniquePeers). > ? > > > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED>wrote: > >> Good call....gb_sets should be faster. Compiled to native it runs faster >> still >> >> 9> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> ok >> >> ________________________________________ >> From: Ahmed Omar [spawn.think@REDACTED] >> Sent: Saturday, March 26, 2011 3:08 PM >> To: Andreas Pauley >> Cc: Evans, Matthew; erlang-questions@REDACTED >> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared >> to Python >> >> using gb_sets instead of sets could decrease eliminations a bit and give u >> some boost. However, i didn't dive deeper into the code or the algorithm. >> >> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED >> <mailto:apauley@REDACTED>> wrote: >> Thanks, I've changed the compile options and this definitely makes it >> somewhat faster: >> >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> >> Strangely the native flag is not documented here: >> http://www.erlang.org/doc/man/compile.html#file-2 >> >> The more interesting improvement would be to decrease the number of >> eliminations performed, but for that I'll have to go deep into the >> code :-) >> >> >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED >> <mailto:mevans@REDACTED>> wrote: >> > Without going deep into the code, one thing to try is compile with the >> native flag (running the tests from the VM shell here): >> > >> > Without native set: >> > 18> c(sudoku). >> > {ok,sudoku} >> > 19> sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> > ok >> > >> > >> > With native set: >> > 20> c(sudoku,[native]). >> > {ok,sudoku} >> > 21> sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> > >> > >> > >> > -----Original Message----- >> > From: erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED> [mailto: >> erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED>] On Behalf Of Andreas Pauley >> > Sent: Friday, March 25, 2011 8:15 AM >> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> Python >> > >> > Hi all, >> > >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >> > solver into Erlang: >> > https://github.com/apauley/sudoku-in-erlang >> > >> > For comparison, my slightly modified version of Norvig's Python code >> > can be found here: >> > https://github.com/apauley/sudoku-by-norvig >> > >> > I like the pattern matching, it was fun to do the entire solution >> > without a single if or case statement :-) >> > Something that bothers me though is that the Python solution seems to >> > be faster, even after I've made the Erlang solution use multiple >> > processors. >> > >> > In order to compare the two solutions I counted the number of >> > eliminations each performed. >> > The eliminate function is the core of the solution, for example >> > assigning a single value to a square is implemented as the elimination >> > of all other values. >> > >> > With the Erlang implementation I get: >> > sudoku-in-erlang$ ./sudoku >> > All tests passed :-) >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> 1770). >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> 1781). >> > >> > And with the Python implementation: >> > sudoku-by-norvig$ ./sudoku.py >> > All tests pass. >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> 648). >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> > >> > So according to the stats above, the Python solution performs less >> > computations when given exactly the same input. >> > The Erlang code is as close to the Python as I could make it, I've >> > done more or less a direct translation of the algorithms used. >> > >> > I suspect that there are some lazy evaluation happening in the Python >> > version, possibly generators, although I haven't pinpointed it yet. >> > >> > How can I improve my Erlang code in this solution? >> > >> > Kind regards, >> > Andreas >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think<http://twitter.com/#!/spawn_think> >> >> > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think <http://twitter.com/#!/spawn_think> > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/2ea42ff0/attachment.htm> From tristan.sloughter@REDACTED Sat Mar 26 21:56:02 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sat, 26 Mar 2011 15:56:02 -0500 Subject: [erlang-questions 40] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> Message-ID: <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> With a few refactorings (https://github.com/tsloughter/sudoku-in-erlang -- note I used ewl_plists just to make the code simpler: https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile with native, hipe o3 it beats python's: $ ./sudoku.py All tests pass. Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) (33059 total eliminations, avg 661.00, median 648, max 830, min 648). Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). $ ./sudoku All tests passed :-) Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). I ran a number of times, those are pretty much what it always comes out to. But I haven't really tried a full refactoring, I think a number of things can be reduced to less iterations. Tristan On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > BTW, beside looking into reducing eleminations (which requires deep dive), > if u do some time profiling using eprof u will find that most of the time > spent actually in the cross functions > sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 > 35024918 [ 0.36] > > > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > >> actually instead of doing >> >> NonUniquePeers = shallow_flatten([S || S <- units(Square)]), >> >> PeerSet = sets:from_list(NonUniquePeers), >> >> PeersWithSelf = sets:to_list(PeerSet), >> >> >> >> can't u just do >> PeersWithSelf = lists:usort(NonUniquePeers). >> >> ? >> >> >> >> >> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED>wrote: >> >>> Good call....gb_sets should be faster. Compiled to native it runs faster >>> still >>> >>> 9> sudoku:print_results("top95.txt", "\n"). >>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >>> 1792). >>> ok >>> >>> ________________________________________ >>> From: Ahmed Omar [spawn.think@REDACTED] >>> Sent: Saturday, March 26, 2011 3:08 PM >>> To: Andreas Pauley >>> Cc: Evans, Matthew; erlang-questions@REDACTED >>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared >>> to Python >>> >>> using gb_sets instead of sets could decrease eliminations a bit and give >>> u some boost. However, i didn't dive deeper into the code or the algorithm. >>> >>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED >>> <mailto:apauley@REDACTED>> wrote: >>> Thanks, I've changed the compile options and this definitely makes it >>> somewhat faster: >>> >>> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >>> >>> Strangely the native flag is not documented here: >>> http://www.erlang.org/doc/man/compile.html#file-2 >>> >>> The more interesting improvement would be to decrease the number of >>> eliminations performed, but for that I'll have to go deep into the >>> code :-) >>> >>> >>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED >>> <mailto:mevans@REDACTED>> wrote: >>> > Without going deep into the code, one thing to try is compile with the >>> native flag (running the tests from the VM shell here): >>> > >>> > Without native set: >>> > 18> c(sudoku). >>> > {ok,sudoku} >>> > 19> sudoku:print_results("top95.txt", "\n"). >>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>> 1797). >>> > ok >>> > >>> > >>> > With native set: >>> > 20> c(sudoku,[native]). >>> > {ok,sudoku} >>> > 21> sudoku:print_results("top95.txt", "\n"). >>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>> 1797). >>> > >>> > >>> > >>> > -----Original Message----- >>> > From: erlang-questions-bounces@REDACTED<mailto: >>> erlang-questions-bounces@REDACTED> [mailto: >>> erlang-questions-bounces@REDACTED<mailto: >>> erlang-questions-bounces@REDACTED>] On Behalf Of Andreas Pauley >>> > Sent: Friday, March 25, 2011 8:15 AM >>> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >>> Python >>> > >>> > Hi all, >>> > >>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >>> > solver into Erlang: >>> > https://github.com/apauley/sudoku-in-erlang >>> > >>> > For comparison, my slightly modified version of Norvig's Python code >>> > can be found here: >>> > https://github.com/apauley/sudoku-by-norvig >>> > >>> > I like the pattern matching, it was fun to do the entire solution >>> > without a single if or case statement :-) >>> > Something that bothers me though is that the Python solution seems to >>> > be faster, even after I've made the Erlang solution use multiple >>> > processors. >>> > >>> > In order to compare the two solutions I counted the number of >>> > eliminations each performed. >>> > The eliminate function is the core of the solution, for example >>> > assigning a single value to a square is implemented as the elimination >>> > of all other values. >>> > >>> > With the Erlang implementation I get: >>> > sudoku-in-erlang$ ./sudoku >>> > All tests passed :-) >>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >>> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min >>> 1770). >>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>> 1797). >>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >>> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min >>> 1781). >>> > >>> > And with the Python implementation: >>> > sudoku-by-norvig$ ./sudoku.py >>> > All tests pass. >>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >>> > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >>> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min >>> 648). >>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >>> > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >>> > >>> > So according to the stats above, the Python solution performs less >>> > computations when given exactly the same input. >>> > The Erlang code is as close to the Python as I could make it, I've >>> > done more or less a direct translation of the algorithms used. >>> > >>> > I suspect that there are some lazy evaluation happening in the Python >>> > version, possibly generators, although I haven't pinpointed it yet. >>> > >>> > How can I improve my Erlang code in this solution? >>> > >>> > Kind regards, >>> > Andreas >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> -- >>> Best Regards, >>> - Ahmed Omar >>> http://nl.linkedin.com/in/adiaa >>> Follow me on twitter >>> @spawn_think<http://twitter.com/#!/spawn_think> >>> >>> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think <http://twitter.com/#!/spawn_think> >> >> > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think <http://twitter.com/#!/spawn_think> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/4b51326f/attachment.htm> From mevans@REDACTED Sat Mar 26 23:33:31 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 26 Mar 2011 18:33:31 -0400 Subject: [erlang-questions 41] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com>, <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> Awesome point, I was going to actually suggest that since this is all calculating static data he just pre-calculates it and puts it in a -define. Or even better, if he is brave he could write a parse transform. Using Ulf's ct_expand module (http://forum.trapexit.org/viewtopic.php?p=20260) I get even better performance: Without the parse transform (but using native): 4> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). ok With a parse transform: 2> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). The code looks like: -compile({parse_transform, ct_expand}). squares() -> %% Returns a list of 81 square names, including "A1" etc. ct_expand:term( [[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] ). col_squares() -> %% All the square names for each column. ct_expand:term( [[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] ). row_squares() -> %% All the square names for each row. ct_expand:term( [[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] ). box_squares() -> %% All the square names for each box. ct_expand:term( [[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- ["123", "456", "789"]] ). ________________________________________ From: Ahmed Omar [spawn.think@REDACTED] Sent: Saturday, March 26, 2011 4:39 PM To: Evans, Matthew Cc: Andreas Pauley; erlang-questions@REDACTED Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python BTW, beside looking into reducing eleminations (which requires deep dive), if u do some time profiling using eprof u will find that most of the time spent actually in the cross functions sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 35024918 [ 0.36] On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED<mailto:spawn.think@REDACTED>> wrote: actually instead of doing NonUniquePeers = shallow_flatten([S || S <- units(Square)]), PeerSet = sets:from_list(NonUniquePeers), PeersWithSelf = sets:to_list(PeerSet), can't u just do PeersWithSelf = lists:usort(NonUniquePeers). ? On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: Good call....gb_sets should be faster. Compiled to native it runs faster still 9> sudoku:print_results("top95.txt", "\n"). Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). ok ________________________________________ From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] Sent: Saturday, March 26, 2011 3:08 PM To: Andreas Pauley Cc: Evans, Matthew; erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto:apauley@REDACTED>>> wrote: Thanks, I've changed the compile options and this definitely makes it somewhat faster: https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb Strangely the native flag is not documented here: http://www.erlang.org/doc/man/compile.html#file-2 The more interesting improvement would be to decrease the number of eliminations performed, but for that I'll have to go deep into the code :-) On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto:mevans@REDACTED>>> wrote: > Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): > > Without native set: > 18> c(sudoku). > {ok,sudoku} > 19> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > ok > > > With native set: > 20> c(sudoku,[native]). > {ok,sudoku} > 21> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > > > > -----Original Message----- > From: erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>> [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley > Sent: Friday, March 25, 2011 8:15 AM > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python > > Hi all, > > In my quest to learn Erlang I've translated Peter Norvig's sudoku > solver into Erlang: > https://github.com/apauley/sudoku-in-erlang > > For comparison, my slightly modified version of Norvig's Python code > can be found here: > https://github.com/apauley/sudoku-by-norvig > > I like the pattern matching, it was fun to do the entire solution > without a single if or case statement :-) > Something that bothers me though is that the Python solution seems to > be faster, even after I've made the Erlang solution use multiple > processors. > > In order to compare the two solutions I counted the number of > eliminations each performed. > The eliminate function is the core of the solution, for example > assigning a single value to a square is implemented as the elimination > of all other values. > > With the Erlang implementation I get: > sudoku-in-erlang$ ./sudoku > All tests passed :-) > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). > > And with the Python implementation: > sudoku-by-norvig$ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > So according to the stats above, the Python solution performs less > computations when given exactly the same input. > The Erlang code is as close to the Python as I could make it, I've > done more or less a direct translation of the algorithms used. > > I suspect that there are some lazy evaluation happening in the Python > version, possibly generators, although I haven't pinpointed it yet. > > How can I improve my Erlang code in this solution? > > Kind regards, > Andreas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> http://erlang.org/mailman/listinfo/erlang-questions -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think<http://twitter.com/#!/spawn_think> -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think<http://twitter.com/#!/spawn_think> -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think<http://twitter.com/#!/spawn_think> From mevans@REDACTED Sat Mar 26 23:45:43 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sat, 26 Mar 2011 18:45:43 -0400 Subject: [erlang-questions 42] Re: HiPE or native....good results, but confused... In-Reply-To: <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com>, <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2C@vvexch.verivue.com> Thanks for the answer. It seems then that there is no negative to using them? Sure, for i/o bound operations you won't see any benefit; but I am assuming there will not be any loss either? /Rant on I do agree with the performance issue - with one BIG caveat: At my own company we are up against a certain amount of FUD when it comes to adopting Erlang. It has proven itself many times over: rapid development times, stability, code swapping, scalability, concurrency and all the other good reasons. However, the performance argument is often used as a negative. I feel like smashing my head against a wall knowing that when all is said and done Erlang will still beat (even well implemented) Java/C/C++ apps of the same complexity (unless they can devote decades of development time in those implementations). But it's often a tough argument to beat. What am I saying? Sure, I agree that it's often fast enough and often times faster than many other languages, but to ensure adoption of the language in more domains we should do whatever optimizations we can... /Rant off ________________________________________ From: Jesper Louis Andersen [jesper.louis.andersen@REDACTED] Sent: Saturday, March 26, 2011 4:34 PM To: Evans, Matthew Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions 33] HiPE or native....good results, but confused... On Sat, Mar 26, 2011 at 20:56, Evans, Matthew <mevans@REDACTED> wrote: > My questions, simply put are: > > 1) When should HiPE be used? When the problem is mostly CPU-bound, you can often gain a factor of 2-5 from using HiPE. Beware there is an overhead in switching to/from HiPE, so if you intermix modules which are HiPE'd and not, you will often incur a penalty in performance. So it is important to compile all used modules into native format. That includes stdlib, so beware of your calls to those. > 2) When shouldn't HiPE be used? For some problems, you are not waiting that much on the CPU, but more time is spent waiting on other subsystems, disk, the database, the network interface and so on. In those cases, the gains from switching to HiPE is neglicible. The programs I work on myself hardly benefits from enabling HiPE. We are talking very little CPU-time spared by doing it. (rant coming) In general, the main reason could be that Erlang is not really built for CPU-bound computation. With current compiler tech, the fastest languages are *clearly* the statically typed languages. Yes, V8+crankshaft is impressive, but it still cannot beat a well-laid-out Haskell or Ocaml program. Common Lisp can be fast, but you need to type the program yourself through type hints to the CL compiler (See CMUCL or SBCL for instance). You are far better off outsourcing the computationally heavy parts. There are at least 4 different ways to get that done in Erlang, (hidden node, port, port_driver, NIF) so you have plenty of options. Then again, people are too obsessed with performance in the first place. Often, it is far more important to think and choose the algorithms and datastructures so they are the right kinds. If you do this well, then your interpreted program can easily be as fast or faster than natively compiled counterparts. This is *especially* true for programs written in weak languages such as C and Java where people don't have access to abstraction features and hence bake everything in with the lowest common denominator. > With native set: > 20> c(sudoku,[native]). Try c(sudoku, [native, {hipe, o3}]) to give it a tad more optimization to work with. It is often faster. -- J. From ulf.wiger@REDACTED Sun Mar 27 01:06:58 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 27 Mar 2011 01:06:58 +0100 Subject: [erlang-questions 43] code:lib_dir and .ez archives Message-ID: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> I packed up EQC Mini and PropEr as a single archive to make it easier to include in an OSS project. http://github.com/esl/run_eqc This works very well, except when compiling modules that require the eqc/proper .hrl files. The -include_lib/1 directive doesn't work if the .hrl is inside an archive. This made me think that the compiler - actually probably file.erl - should support paths where a component is an .ez archive, and do the sensible thing (act as if it were a real path). But first, I noticed that code:lib_dir/1 acts very strangely when the application is loaded from an .ez archive (see below). Is this deliberate, or is it a bug? BR, Ulf W $ erl -pa run_proper.ez/proper/ebin Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> code:which(proper). "run_proper.ez/proper/ebin/proper.beam" 2> code:lib_dir(proper). "./run_proper" 3> {ok,Z} = zip:zip_open("run_proper.ez",[memory]). {ok,<0.35.0>} 4> zip:zip_get("proper/include/proper.hrl",Z). {ok,{"proper/include/proper.hrl", <<"%%% Copyright 2010 Manolis Papadakis (manopapad@REDACTED)\n%%% and Kostis Sagonas (kostis"...>>}} 5> zip:zip_get("proper/ebin/proper.app",Z). {ok,{"proper/ebin/proper.app", <<"{application,proper,\n [{description,\"A QuickCheck-inspired property-based testing tool f"...>>}} Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From olivier.girondel@REDACTED Sun Mar 27 04:26:21 2011 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Sun, 27 Mar 2011 04:26:21 +0200 Subject: [erlang-questions 44] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> Message-ID: <AANLkTimM=55XqgX7LyvOtzsYMPaQRobMWQV7DRLvnOC9@mail.gmail.com> Surely totally unrelated, but the words "erlang" and "sudoku" always make me remember of http://www.erlang-solutions.com/section/47/2006-competition#FirstPrizeB :) -- Olivier From michael.eugene.turner@REDACTED Sun Mar 27 09:24:00 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 27 Mar 2011 16:24:00 +0900 Subject: [erlang-questions 45] wxWidgets draft/stub tutorial available Message-ID: <AANLkTinaYEHLpdssx3WCHiTa6XdtVa-2UevhzXVQ+1bq@mail.gmail.com> I've thrown together the beginnings of a tutorial for wxWidgets in Erlang. http://www.idiom.com/~turner/wxtut/wxwidgets.html My goal is not just to give wx newbies a running start. I'm also exploring a Tcl/Tk style of desktop app building in Erlang. However, the Erlang shell, wx's rather slavish mimicking of the wxWidgets C++ API, and the process architecture of wx as an "application" have all hindered me somewhat, for the "build simple GUIs mostly from the command line" goal. For example, there's no way to define macros in the Erlang shell, much less a way to read macro definitions from a header file; and wx relies heavily on macro definitions. Making the reader look up the macro values in the headers and supply them as "magic numbers" in the Erlang shell feels distinctly sub-optimal. As well, wx graphics runs in its own process. To add callbacks defined as funs from the command line seems awkward across the IPC barrier for all but the simplest callbacks -- though of course it's a cool thing about Erlang that you can do this at all. There might be easier ways than I can currently imagine for reaching the goals of this tutorial. For example, the next "chapter" (or two) might to take the reader through the process of building a GUI for a kind of specialized mini-shell that "knows" (from parsing the wx header file(s)) the wx macro definitions -- though I must admit I've only begun to think about how it might work. In part because Erlang/OTP contains excellent parsing tools for Erlang itself, adding this kind of thing is much easier than in some other programming languages. In any case, feedback is very welcome. I'm curious about the level of interest out there in any continuing work on this tutorial, since the current draft could potentially be the first of a dozen of more chapters -- wxWidgets is a big topic. If I hear from even a handful of people who worked through this draft and found themselves thirsty for more, that's enough of an audience for me. The repo is here: https://github.com/Yakushima/wxtut -michael turner -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/f70769cd/attachment.htm> From ulf.wiger@REDACTED Sun Mar 27 10:07:27 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 27 Mar 2011 10:07:27 +0200 Subject: [erlang-questions 46] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTimM=55XqgX7LyvOtzsYMPaQRobMWQV7DRLvnOC9@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <AANLkTimM=55XqgX7LyvOtzsYMPaQRobMWQV7DRLvnOC9@mail.gmail.com> Message-ID: <61C4AE2F-FF77-47CA-B509-25674F8C5238@erlang-solutions.com> On 27 Mar 2011, at 04:26, Olivier Girondel wrote: > http://www.erlang-solutions.com/section/47/2006-competition#FirstPrizeB ?or this one: http://www.erlang-solutions.com/section/48/2005-competition#SecondPrize Surely the most beautiful Erlang program ever written! BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/0610cdf0/attachment.htm> From ulf.wiger@REDACTED Sun Mar 27 10:13:37 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 27 Mar 2011 10:13:37 +0200 Subject: [erlang-questions 47] Re: wxWidgets draft/stub tutorial available In-Reply-To: <AANLkTinaYEHLpdssx3WCHiTa6XdtVa-2UevhzXVQ+1bq@mail.gmail.com> References: <AANLkTinaYEHLpdssx3WCHiTa6XdtVa-2UevhzXVQ+1bq@mail.gmail.com> Message-ID: <44E44C1B-20E5-4415-869D-D9BA2C392547@erlang-solutions.com> On 27 Mar 2011, at 09:24, Michael Turner wrote: > However, the Erlang shell, wx's rather slavish mimicking of the wxWidgets C++ API, and the process architecture of wx as an "application" have all hindered me somewhat, for the "build simple GUIs mostly from the command line" goal. > > For example, there's no way to define macros in the Erlang shell, much less a way to read macro definitions from a header file; and wx relies heavily on macro definitions. Making the reader look up the macro values in the headers and supply them as "magic numbers" in the Erlang shell feels distinctly sub-optimal. This gives me an excuse to mention my old shell extension hack, http://ulf.wiger.net/weblog/2007/11/21/extending-the-erlang-shell-part-2/ which allowed for a way to define modules from the shell, support alternative parsers, and also import macros in the same way as the current shell imports record definitions. The code would need to be ported to R14B, and included an extension to eep.erl, among other things. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/476e0a80/attachment.htm> From thomasl_erlang@REDACTED Sun Mar 27 14:46:05 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sun, 27 Mar 2011 05:46:05 -0700 (PDT) Subject: [erlang-questions 48] Re: HiPE or native....good results, but confused... In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> Message-ID: <706425.39560.qm@web111416.mail.gq1.yahoo.com> ----- Original Message ---- > From: "Evans, Matthew" <mevans@REDACTED> > To: "erlang-questions@REDACTED" <erlang-questions@REDACTED> > Sent: Sat, March 26, 2011 8:56:12 PM > Subject: [erlang-questions 33] HiPE or native....good results, but confused... > > Hi, > > If anyone has been following the Sudoku email thread started by Andreas, I >suggested he looked at compiling his code with the "native" flag set. With his >app, and some common Erlang modules compiled to be native (espc. lists, sets >and dict) I got a 2x performance improvement (see below). > > I hadn't really played with HiPE (native) much, as I was always under the >impression that performance gains were negligible. Clearly, for this use case, >the gains are far from negligible - a 2x boost is not something to ignore. Actually, applications where you're _not_ spending serious time in BIFs, NIFs, drivers, scheduling, message passing etc, is one of the good cases for Hipe. Basically, the compiler can only optimize what it can "see", and there is usually not much to be done with C calls into the runtime system. But that doesn't mean all is lost. From what we heard on this list a couple of weeks ago, dialyzer seems to get good speedup from native compilation, for instance. Similar programs (e.g., symbolic processing, compilers, ...) would also be interesting candidates for Hipe. Best regards, Thomas From krab@REDACTED Sun Mar 27 15:28:02 2011 From: krab@REDACTED (Kresten Krab Thorup) Date: Sun, 27 Mar 2011 15:28:02 +0200 Subject: [erlang-questions 49] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com>, <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> Message-ID: <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> For reference, I tried to run this with Erjang and R14B01 (with BEAN and HiPE/o3) on my machine ... Oh, and I changed the square names to be atoms in stead of [X,Y] pairs, which speeds up things quite a lot too ... e.g. : squares() -> %% Returns a list of 81 square names, including "A1" etc. [erlang:list_to_atom([X,Y]) || X <- "ABCDEFGHI", Y <- "123456789"]. BEAM > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.678921 secs (73.65 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 4.373277 secs (21.72 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.198780 secs (55.34 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok Erjang > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.653891 secs (76.47 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 4.354102 secs (21.82 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.208266 secs (52.82 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok r14b01 / HiPE / o3 > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.611918 secs (81.71 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 3.759281 secs (25.27 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.169039 secs (65.07 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok This version does not use pmap (Erjang's scheduler still needs some work), but does use atoms for square names, and ct_expand. Cheers, Kresten On Mar 26, 2011, at 23:33 , Evans, Matthew wrote: > Awesome point, I was going to actually suggest that since this is all calculating static data he just pre-calculates it and puts it in a -define. Or even better, if he is brave he could write a parse transform. > > Using Ulf's ct_expand module (http://forum.trapexit.org/viewtopic.php?p=20260) I get even better performance: > > Without the parse transform (but using native): > > 4> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > > With a parse transform: > > 2> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > > > The code looks like: > > -compile({parse_transform, ct_expand}). > > > squares() -> > %% Returns a list of 81 square names, including "A1" etc. > ct_expand:term( > [[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] > ). > col_squares() -> > %% All the square names for each column. > ct_expand:term( > [[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] > ). > row_squares() -> > %% All the square names for each row. > ct_expand:term( > [[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] > ). > box_squares() -> > %% All the square names for each box. > ct_expand:term( > [[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- ["123", "456", "789"]] > ). > > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED] > Sent: Saturday, March 26, 2011 4:39 PM > To: Evans, Matthew > Cc: Andreas Pauley; erlang-questions@REDACTED > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > BTW, beside looking into reducing eleminations (which requires deep dive), if u do some time profiling using eprof u will find that most of the time spent actually in the cross functions > sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 35024918 [ 0.36] > > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED<mailto:spawn.think@REDACTED>> wrote: > actually instead of doing > > > > NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > > > > PeerSet = sets:from_list(NonUniquePeers), > > > > PeersWithSelf = sets:to_list(PeerSet), > > > > > > > > > can't u just do > > > PeersWithSelf = lists:usort(NonUniquePeers). > > > > ? > > > > > > > > > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > Good call....gb_sets should be faster. Compiled to native it runs faster still > > 9> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] > Sent: Saturday, March 26, 2011 3:08 PM > To: Andreas Pauley > Cc: Evans, Matthew; erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto:apauley@REDACTED>>> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto:mevans@REDACTED>>> wrote: >> Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): >> >> Without native set: >> 18> c(sudoku). >> {ok,sudoku} >> 19> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> ok >> >> >> With native set: >> 20> c(sudoku,[native]). >> {ok,sudoku} >> 21> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> >> >> >> -----Original Message----- >> From: erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>> [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley >> Sent: Friday, March 25, 2011 8:15 AM >> To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python >> >> Hi all, >> >> In my quest to learn Erlang I've translated Peter Norvig's sudoku >> solver into Erlang: >> https://github.com/apauley/sudoku-in-erlang >> >> For comparison, my slightly modified version of Norvig's Python code >> can be found here: >> https://github.com/apauley/sudoku-by-norvig >> >> I like the pattern matching, it was fun to do the entire solution >> without a single if or case statement :-) >> Something that bothers me though is that the Python solution seems to >> be faster, even after I've made the Erlang solution use multiple >> processors. >> >> In order to compare the two solutions I counted the number of >> eliminations each performed. >> The eliminate function is the core of the solution, for example >> assigning a single value to a square is implemented as the elimination >> of all other values. >> >> With the Erlang implementation I get: >> sudoku-in-erlang$ ./sudoku >> All tests passed :-) >> Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). >> Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). >> >> And with the Python implementation: >> sudoku-by-norvig$ ./sudoku.py >> All tests pass. >> Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). >> Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> >> So according to the stats above, the Python solution performs less >> computations when given exactly the same input. >> The Erlang code is as close to the Python as I could make it, I've >> done more or less a direct translation of the algorithms used. >> >> I suspect that there are some lazy evaluation happening in the Python >> version, possibly generators, although I haven't pinpointed it yet. >> >> How can I improve my Erlang code in this solution? >> >> Kind regards, >> Andreas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ericbmerritt@REDACTED Sun Mar 27 15:57:43 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Sun, 27 Mar 2011 08:57:43 -0500 Subject: [erlang-questions 50] Re: code:lib_dir and .ez archives In-Reply-To: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> References: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> Message-ID: <AANLkTin-_4aNeJqE-SZ-kCx9rn6wugXd43DDKySs7B7S@mail.gmail.com> There are a few problems like this in the escript stuff. For example, there isn't any way to do releases in escript/ez archives. I think the main problem is that the escript stuff just hasn't been fully integrated into the erlang world. It seems like its 80% of the way there but that additional 20% is still hanging out. I would love to see those extra cases covered. On Sat, Mar 26, 2011 at 7:06 PM, Ulf Wiger <ulf.wiger@REDACTED> wrote: > > I packed up EQC Mini and PropEr as a single archive to make it easier to include in an OSS project. > > http://github.com/esl/run_eqc > > This works very well, except when compiling modules that require the eqc/proper .hrl files. The -include_lib/1 directive doesn't work if the .hrl is inside an archive. > > This made me think that the compiler - actually probably file.erl - should support paths where a component is an .ez archive, and do the sensible thing (act as if it were a real path). > > But first, I noticed that code:lib_dir/1 acts very strangely when the application is loaded from an .ez archive (see below). > > Is this deliberate, or is it a bug? > > BR, > Ulf W > > $ erl -pa run_proper.ez/proper/ebin > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 ?(abort with ^G) > 1> code:which(proper). > "run_proper.ez/proper/ebin/proper.beam" > 2> code:lib_dir(proper). > "./run_proper" > 3> {ok,Z} = zip:zip_open("run_proper.ez",[memory]). > {ok,<0.35.0>} > 4> zip:zip_get("proper/include/proper.hrl",Z). > {ok,{"proper/include/proper.hrl", > ? ? <<"%%% Copyright 2010 Manolis Papadakis (manopapad@REDACTED)\n%%% ? ? ? ? ? ?and Kostis Sagonas (kostis"...>>}} > 5> zip:zip_get("proper/ebin/proper.app",Z). > {ok,{"proper/ebin/proper.app", > ? ? <<"{application,proper,\n ? ? ? ? ? ? [{description,\"A QuickCheck-inspired property-based testing tool f"...>>}} > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mevans@REDACTED Sun Mar 27 17:49:12 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sun, 27 Mar 2011 11:49:12 -0400 Subject: [erlang-questions 51] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com>, <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com>, <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2D@vvexch.verivue.com> Good call on using atoms instead of lists. I wouldn't have imagined it would be that much faster. So on my computer the new Erlang implementation is much faster than python: Erlang with parse transforms and HiPE: [mevans@REDACTED ~]$ ./sudoku expanding... expanding... expanding... expanding... Solved 50 of 50 puzzles from easy50.txt in 0.059767 secs (836.58 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 0.344457 secs (275.80 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.025747 secs (427.23 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). Python: [mevans@REDACTED ~]$ ./sudoku.py All tests pass. Solved 50 of 50 puzzles from easy50.txt in 0.530000 secs (94.34 Hz) (33059 total eliminations, avg 661.00, median 648, max 830, min 648). Solved 95 of 95 puzzles from top95.txt in 3.980000 secs (23.87 Hz) (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). Solved 11 of 11 puzzles from hardest.txt in 0.150000 secs (73.33 Hz) (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). [mevans@REDACTED ~]$ The initial implementation in Erlang: [mevans@REDACTED ~]$ ./sudoku ./sudoku:4: Warning: variable 'Args' is unused Solved 50 of 50 puzzles from easy50.txt in 0.492551 secs (101.51 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 3.120420 secs (30.44 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.147977 secs (74.34 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ________________________________________ From: Kresten Krab Thorup [krab@REDACTED] Sent: Sunday, March 27, 2011 9:28 AM To: Evans, Matthew Cc: Ahmed Omar; erlang-questions@REDACTED; Andreas Pauley Subject: Re: [erlang-questions 41] Re: A sudoku solver in Erlang compared to Python For reference, I tried to run this with Erjang and R14B01 (with BEAN and HiPE/o3) on my machine ... Oh, and I changed the square names to be atoms in stead of [X,Y] pairs, which speeds up things quite a lot too ... e.g. : squares() -> %% Returns a list of 81 square names, including "A1" etc. [erlang:list_to_atom([X,Y]) || X <- "ABCDEFGHI", Y <- "123456789"]. BEAM > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.678921 secs (73.65 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 4.373277 secs (21.72 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.198780 secs (55.34 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok Erjang > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.653891 secs (76.47 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 4.354102 secs (21.82 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.208266 secs (52.82 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok r14b01 / HiPE / o3 > sudoku_main:run_solver(["solve"]). Solved 50 of 50 puzzles from easy50.txt in 0.611918 secs (81.71 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 3.759281 secs (25.27 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.169039 secs (65.07 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). ok This version does not use pmap (Erjang's scheduler still needs some work), but does use atoms for square names, and ct_expand. Cheers, Kresten On Mar 26, 2011, at 23:33 , Evans, Matthew wrote: > Awesome point, I was going to actually suggest that since this is all calculating static data he just pre-calculates it and puts it in a -define. Or even better, if he is brave he could write a parse transform. > > Using Ulf's ct_expand module (http://forum.trapexit.org/viewtopic.php?p=20260) I get even better performance: > > Without the parse transform (but using native): > > 4> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > > With a parse transform: > > 2> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > > > The code looks like: > > -compile({parse_transform, ct_expand}). > > > squares() -> > %% Returns a list of 81 square names, including "A1" etc. > ct_expand:term( > [[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] > ). > col_squares() -> > %% All the square names for each column. > ct_expand:term( > [[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] > ). > row_squares() -> > %% All the square names for each row. > ct_expand:term( > [[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] > ). > box_squares() -> > %% All the square names for each box. > ct_expand:term( > [[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- ["123", "456", "789"]] > ). > > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED] > Sent: Saturday, March 26, 2011 4:39 PM > To: Evans, Matthew > Cc: Andreas Pauley; erlang-questions@REDACTED > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > BTW, beside looking into reducing eleminations (which requires deep dive), if u do some time profiling using eprof u will find that most of the time spent actually in the cross functions > sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 35024918 [ 0.36] > > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED<mailto:spawn.think@REDACTED>> wrote: > actually instead of doing > > > > NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > > > > PeerSet = sets:from_list(NonUniquePeers), > > > > PeersWithSelf = sets:to_list(PeerSet), > > > > > > > > > can't u just do > > > PeersWithSelf = lists:usort(NonUniquePeers). > > > > ? > > > > > > > > > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > Good call....gb_sets should be faster. Compiled to native it runs faster still > > 9> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] > Sent: Saturday, March 26, 2011 3:08 PM > To: Andreas Pauley > Cc: Evans, Matthew; erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto:apauley@REDACTED>>> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto:mevans@REDACTED>>> wrote: >> Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): >> >> Without native set: >> 18> c(sudoku). >> {ok,sudoku} >> 19> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> ok >> >> >> With native set: >> 20> c(sudoku,[native]). >> {ok,sudoku} >> 21> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> >> >> >> -----Original Message----- >> From: erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>> [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley >> Sent: Friday, March 25, 2011 8:15 AM >> To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python >> >> Hi all, >> >> In my quest to learn Erlang I've translated Peter Norvig's sudoku >> solver into Erlang: >> https://github.com/apauley/sudoku-in-erlang >> >> For comparison, my slightly modified version of Norvig's Python code >> can be found here: >> https://github.com/apauley/sudoku-by-norvig >> >> I like the pattern matching, it was fun to do the entire solution >> without a single if or case statement :-) >> Something that bothers me though is that the Python solution seems to >> be faster, even after I've made the Erlang solution use multiple >> processors. >> >> In order to compare the two solutions I counted the number of >> eliminations each performed. >> The eliminate function is the core of the solution, for example >> assigning a single value to a square is implemented as the elimination >> of all other values. >> >> With the Erlang implementation I get: >> sudoku-in-erlang$ ./sudoku >> All tests passed :-) >> Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> (93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). >> Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> (32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). >> >> And with the Python implementation: >> sudoku-by-norvig$ ./sudoku.py >> All tests pass. >> Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). >> Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> >> So according to the stats above, the Python solution performs less >> computations when given exactly the same input. >> The Erlang code is as close to the Python as I could make it, I've >> done more or less a direct translation of the algorithms used. >> >> I suspect that there are some lazy evaluation happening in the Python >> version, possibly generators, although I haven't pinpointed it yet. >> >> How can I improve my Erlang code in this solution? >> >> Kind regards, >> Andreas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andrew@REDACTED Sun Mar 27 18:58:13 2011 From: andrew@REDACTED (Andrew Thompson) Date: Sun, 27 Mar 2011 12:58:13 -0400 Subject: [erlang-questions 52] Re: HiPE or native....good results, but confused... In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2C@vvexch.verivue.com> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2C@vvexch.verivue.com> Message-ID: <20110327165813.GL20461@hijacked.us> On Sat, Mar 26, 2011 at 06:45:43PM -0400, Evans, Matthew wrote: > Thanks for the answer. It seems then that there is no negative to using them? Sure, for i/o bound operations you won't see any benefit; but I am assuming there will not be any loss either? > I've found that using HiPE triggers all kinds of insane behaviour or segfaults when I try to use it, so I've given up considering it a valid optimization approach. Andrew From kostis@REDACTED Sun Mar 27 19:06:28 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 27 Mar 2011 20:06:28 +0300 Subject: [erlang-questions 53] Re: HiPE or native....good results, but confused... In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> Message-ID: <4D8F6E94.80607@cs.ntua.gr> Evans, Matthew wrote: > > I hadn't really played with HiPE (native) much, as I was always under the impression that performance gains were negligible. Clearly, for this use case, the gains are far from negligible - a 2x boost is not something to ignore. > > My questions, simply put are: > > 1) When should HiPE be used? Well, my opinion is that you should always try it and decide based on the performance benefits you get on your application. Also, in case you were not aware of this there is a configure option (--enable-native-libs) that compiles some applications (erts, kernel, stdlib, syntax_tools and dialyzer) to native code. > 2) When shouldn't HiPE be used? When you do not get any considerable performance gains from using it. > 3) What are the disadvantages of using HiPE? You need more code space, code is currently not unloaded and often you also lose some precision in the debugging/crash_dump information. > 4) Shouldn't "team Erlang" be doing more to promote its use? I will let others reply/comment on this one. > 5) Are any HiPE changes on the road-map? The HiPE compiler is pretty stable; the only bug I am aware of is an off by one error in binary pattern matching which has already been fixed and the fix will appear on 'dev' soon. (For comparison, incidentally the R14B02 BEAM compiler also had a bug in binary pattern matching which has been fixed two days ago.) Such is life, I guess... The only thing to do about it is to hope that the Erlang community will help by reporting whatever issues they run across because both the BEAM and the HiPE compiler developers are currently committed to fixing them. Besides support, we have developed a PowerPC 64 backend for HiPE which has not been included to the R14B02 release due to some miscommunication with the OTP team. Anybody who is interested in it can try it from the 'pu' branch of Erlang/OTP. In addition, we are currently working on an LLVM backend for HiPE. We'll see what the results will look like. Kostis From kostis@REDACTED Sun Mar 27 19:10:40 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 27 Mar 2011 20:10:40 +0300 Subject: [erlang-questions 54] Re: HiPE or native....good results, but confused... In-Reply-To: <20110327165813.GL20461@hijacked.us> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2C@vvexch.verivue.com> <20110327165813.GL20461@hijacked.us> Message-ID: <4D8F6F90.8000003@cs.ntua.gr> Andrew Thompson wrote: > On Sat, Mar 26, 2011 at 06:45:43PM -0400, Evans, Matthew wrote: >> Thanks for the answer. It seems then that there is no negative to using them? Sure, for i/o bound operations you won't see any benefit; but I am assuming there will not be any loss either? >> > I've found that using HiPE triggers all kinds of insane behaviour or > segfaults when I try to use it, so I've given up considering it a valid > optimization approach. You are of course free to do whatever you want, but wouldn't send a bug report (and a test case for) these problems be a better approach than sending just this mail to the mailing list? Kostis From steven.charles.davis@REDACTED Sun Mar 27 19:47:46 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 27 Mar 2011 10:47:46 -0700 (PDT) Subject: [erlang-questions 55] Re: code:lib_dir and .ez archives In-Reply-To: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> References: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> Message-ID: <1301248066671-3409584.post@n4.nabble.com> Are the results different if you rename run_proper.ez to proper-0.1.ez ? -- View this message in context: http://erlang.2086793.n4.nabble.com/erlang-questions-43-code-lib-dir-and-ez-archives-tp3408624p3409584.html Sent from the Erlang Questions mailing list archive at Nabble.com. From spawn.think@REDACTED Sun Mar 27 20:56:55 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sun, 27 Mar 2011 20:56:55 +0200 Subject: [erlang-questions 56] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2D@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2D@vvexch.verivue.com> Message-ID: <AANLkTint=W+RaupLNXkVmarWgRKM_sMSc5aUa5fR+NDa@mail.gmail.com> Regarding the eliminations number, i believe you have a bug in counting them (correct me if i'm wrong. You shouldn't increment the counter until an actual elimination occurs. -eliminate(Puzzle, [Square|T], Digit) -> +eliminate({Dict, Count}, [Square|T], Digit) -> %% Eliminate the specified Digit from all specified Squares. + Puzzle = {Dict, Count+1}, eliminate({ValuesDict, Eliminations}, Square, Digit, NewValues, _) -> NewDict = dict:store(Square, NewValues, ValuesDict), - NewPuzzle = peer_eliminate({NewDict, Eliminations+1}, Square, NewValues), + NewPuzzle = peer_eliminate({NewDict, Eliminations}, Square, NewValues), On Sun, Mar 27, 2011 at 5:49 PM, Evans, Matthew <mevans@REDACTED> wrote: > > Good call on using atoms instead of lists. I wouldn't have imagined it > would be that much faster. So on my computer the new Erlang implementation > is much faster than python: > > Erlang with parse transforms and HiPE: > > [mevans@REDACTED ~]$ ./sudoku > expanding... > expanding... > expanding... > expanding... > Solved 50 of 50 puzzles from easy50.txt in 0.059767 secs (836.58 Hz) > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 0.344457 secs (275.80 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.025747 secs (427.23 Hz) > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > > > Python: > > [mevans@REDACTED ~]$ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.530000 secs (94.34 Hz) > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 3.980000 secs (23.87 Hz) > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.150000 secs (73.33 Hz) > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > [mevans@REDACTED ~]$ > > > The initial implementation in Erlang: > > [mevans@REDACTED ~]$ ./sudoku > ./sudoku:4: Warning: variable 'Args' is unused > Solved 50 of 50 puzzles from easy50.txt in 0.492551 secs (101.51 Hz) > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 3.120420 secs (30.44 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.147977 secs (74.34 Hz) > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > > ________________________________________ > From: Kresten Krab Thorup [krab@REDACTED] > Sent: Sunday, March 27, 2011 9:28 AM > To: Evans, Matthew > Cc: Ahmed Omar; erlang-questions@REDACTED; Andreas Pauley > Subject: Re: [erlang-questions 41] Re: A sudoku solver in Erlang compared > to Python > > For reference, I tried to run this with Erjang and R14B01 (with BEAN and > HiPE/o3) on my machine ... > > Oh, and I changed the square names to be atoms in stead of [X,Y] pairs, > which speeds up things quite a lot too ... e.g. : > > squares() -> > %% Returns a list of 81 square names, including "A1" etc. > [erlang:list_to_atom([X,Y]) || X <- "ABCDEFGHI", Y <- "123456789"]. > > > BEAM > > sudoku_main:run_solver(["solve"]). > Solved 50 of 50 puzzles from easy50.txt in 0.678921 secs (73.65 Hz) > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 4.373277 secs (21.72 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.198780 secs (55.34 Hz) > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > ok > > Erjang > > sudoku_main:run_solver(["solve"]). > Solved 50 of 50 puzzles from easy50.txt in 0.653891 secs (76.47 Hz) > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 4.354102 secs (21.82 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.208266 secs (52.82 Hz) > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > ok > > r14b01 / HiPE / o3 > > sudoku_main:run_solver(["solve"]). > Solved 50 of 50 puzzles from easy50.txt in 0.611918 secs (81.71 Hz) > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 3.759281 secs (25.27 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.169039 secs (65.07 Hz) > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > ok > > This version does not use pmap (Erjang's scheduler still needs some work), > but does use atoms for square names, and ct_expand. > > Cheers, > > Kresten > > > On Mar 26, 2011, at 23:33 , Evans, Matthew wrote: > > > Awesome point, I was going to actually suggest that since this is all > calculating static data he just pre-calculates it and puts it in a -define. > Or even better, if he is brave he could write a parse transform. > > > > Using Ulf's ct_expand module ( > http://forum.trapexit.org/viewtopic.php?p=20260) I get even better > performance: > > > > Without the parse transform (but using native): > > > > 4> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) > > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > > ok > > > > > > With a parse transform: > > > > 2> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) > > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > > > > > > The code looks like: > > > > -compile({parse_transform, ct_expand}). > > > > > > squares() -> > > %% Returns a list of 81 square names, including "A1" etc. > > ct_expand:term( > > [[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] > > ). > > col_squares() -> > > %% All the square names for each column. > > ct_expand:term( > > [[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] > > ). > > row_squares() -> > > %% All the square names for each row. > > ct_expand:term( > > [[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] > > ). > > box_squares() -> > > %% All the square names for each box. > > ct_expand:term( > > [[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- > ["123", "456", "789"]] > > ). > > > > > > ________________________________________ > > From: Ahmed Omar [spawn.think@REDACTED] > > Sent: Saturday, March 26, 2011 4:39 PM > > To: Evans, Matthew > > Cc: Andreas Pauley; erlang-questions@REDACTED > > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared > to Python > > > > BTW, beside looking into reducing eleminations (which requires deep > dive), if u do some time profiling using eprof u will find that most of the > time spent actually in the cross functions > > sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 > 35024918 [ 0.36] > > > > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED > <mailto:spawn.think@REDACTED>> wrote: > > actually instead of doing > > > > > > > > NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > > > > > > > > PeerSet = sets:from_list(NonUniquePeers), > > > > > > > > PeersWithSelf = sets:to_list(PeerSet), > > > > > > > > > > > > > > > > > > can't u just do > > > > > > PeersWithSelf = lists:usort(NonUniquePeers). > > > > > > > > ? > > > > > > > > > > > > > > > > > > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED > <mailto:mevans@REDACTED>> wrote: > > Good call....gb_sets should be faster. Compiled to native it runs faster > still > > > > 9> sudoku:print_results("top95.txt", "\n"). > > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > > ok > > > > ________________________________________ > > From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] > > Sent: Saturday, March 26, 2011 3:08 PM > > To: Andreas Pauley > > Cc: Evans, Matthew; erlang-questions@REDACTED<mailto: > erlang-questions@REDACTED> > > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared > to Python > > > > using gb_sets instead of sets could decrease eliminations a bit and give > u some boost. However, i didn't dive deeper into the code or the algorithm. > > > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED > <mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto: > apauley@REDACTED>>> wrote: > > Thanks, I've changed the compile options and this definitely makes it > > somewhat faster: > > > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > > > Strangely the native flag is not documented here: > > http://www.erlang.org/doc/man/compile.html#file-2 > > > > The more interesting improvement would be to decrease the number of > > eliminations performed, but for that I'll have to go deep into the > > code :-) > > > > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED > <mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto: > mevans@REDACTED>>> wrote: > >> Without going deep into the code, one thing to try is compile with the > native flag (running the tests from the VM shell here): > >> > >> Without native set: > >> 18> c(sudoku). > >> {ok,sudoku} > >> 19> sudoku:print_results("top95.txt", "\n"). > >> Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > >> ok > >> > >> > >> With native set: > >> 20> c(sudoku,[native]). > >> {ok,sudoku} > >> 21> sudoku:print_results("top95.txt", "\n"). > >> Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > >> > >> > >> > >> -----Original Message----- > >> From: erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED><mailto: > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>> [mailto: > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED><mailto: > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley > >> Sent: Friday, March 25, 2011 8:15 AM > >> To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED > ><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > >> Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > Python > >> > >> Hi all, > >> > >> In my quest to learn Erlang I've translated Peter Norvig's sudoku > >> solver into Erlang: > >> https://github.com/apauley/sudoku-in-erlang > >> > >> For comparison, my slightly modified version of Norvig's Python code > >> can be found here: > >> https://github.com/apauley/sudoku-by-norvig > >> > >> I like the pattern matching, it was fun to do the entire solution > >> without a single if or case statement :-) > >> Something that bothers me though is that the Python solution seems to > >> be faster, even after I've made the Erlang solution use multiple > >> processors. > >> > >> In order to compare the two solutions I counted the number of > >> eliminations each performed. > >> The eliminate function is the core of the solution, for example > >> assigning a single value to a square is implemented as the elimination > >> of all other values. > >> > >> With the Erlang implementation I get: > >> sudoku-in-erlang$ ./sudoku > >> All tests passed :-) > >> Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > >> (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > 1770). > >> Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > 1797). > >> Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > >> (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > 1781). > >> > >> And with the Python implementation: > >> sudoku-by-norvig$ ./sudoku.py > >> All tests pass. > >> Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > >> (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > >> Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > >> (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > >> Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > >> (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > >> > >> So according to the stats above, the Python solution performs less > >> computations when given exactly the same input. > >> The Erlang code is as close to the Python as I could make it, I've > >> done more or less a direct translation of the algorithms used. > >> > >> I suspect that there are some lazy evaluation happening in the Python > >> version, possibly generators, although I haven't pinpointed it yet. > >> > >> How can I improve my Erlang code in this solution? > >> > >> Kind regards, > >> Andreas > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto: > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto: > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > Best Regards, > > - Ahmed Omar > > http://nl.linkedin.com/in/adiaa > > Follow me on twitter > > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > > > > > > -- > > Best Regards, > > - Ahmed Omar > > http://nl.linkedin.com/in/adiaa > > Follow me on twitter > > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > > > > > > -- > > Best Regards, > > - Ahmed Omar > > http://nl.linkedin.com/in/adiaa > > Follow me on twitter > > @spawn_think<http://twitter.com/#!/spawn_think> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/75226450/attachment.htm> From spawn.think@REDACTED Sun Mar 27 20:58:33 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sun, 27 Mar 2011 20:58:33 +0200 Subject: [erlang-questions 57] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTint=W+RaupLNXkVmarWgRKM_sMSc5aUa5fR+NDa@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2D@vvexch.verivue.com> <AANLkTint=W+RaupLNXkVmarWgRKM_sMSc5aUa5fR+NDa@mail.gmail.com> Message-ID: <AANLkTintw=pmBSrwV5HNMpvWw4Uh2HPJcGTdCG=VNrrT@mail.gmail.com> Sorry diff was inverted :) -eliminate({Dict, Count}, [Square|T], Digit) -> +eliminate(Puzzle, [Square|T], Digit) -> %% Eliminate the specified Digit from all specified Squares. - Puzzle = {Dict, Count+1}, eliminate({ValuesDict, Eliminations}, Square, Digit, NewValues, _) -> NewDict = dict:store(Square, NewValues, ValuesDict), - NewPuzzle = peer_eliminate({NewDict, Eliminations}, Square, NewValues), + NewPuzzle = peer_eliminate({NewDict, Eliminations+1}, Square, NewValues), On Sun, Mar 27, 2011 at 8:56 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > Regarding the eliminations number, i believe you have a bug in counting > them (correct me if i'm wrong. You shouldn't increment the counter until an > actual elimination occurs. > > -eliminate(Puzzle, [Square|T], Digit) -> > +eliminate({Dict, Count}, [Square|T], Digit) -> > %% Eliminate the specified Digit from all specified Squares. > + Puzzle = {Dict, Count+1}, > > eliminate({ValuesDict, Eliminations}, Square, Digit, NewValues, _) -> > NewDict = dict:store(Square, NewValues, ValuesDict), > - NewPuzzle = peer_eliminate({NewDict, Eliminations+1}, Square, > NewValues), > + NewPuzzle = peer_eliminate({NewDict, Eliminations}, Square, > NewValues), > > > > On Sun, Mar 27, 2011 at 5:49 PM, Evans, Matthew <mevans@REDACTED>wrote: > >> >> Good call on using atoms instead of lists. I wouldn't have imagined it >> would be that much faster. So on my computer the new Erlang implementation >> is much faster than python: >> >> Erlang with parse transforms and HiPE: >> >> [mevans@REDACTED ~]$ ./sudoku >> expanding... >> expanding... >> expanding... >> expanding... >> Solved 50 of 50 puzzles from easy50.txt in 0.059767 secs (836.58 Hz) >> (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). >> Solved 95 of 95 puzzles from top95.txt in 0.344457 secs (275.80 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> Solved 11 of 11 puzzles from hardest.txt in 0.025747 secs (427.23 Hz) >> (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). >> >> >> Python: >> >> [mevans@REDACTED ~]$ ./sudoku.py >> All tests pass. >> Solved 50 of 50 puzzles from easy50.txt in 0.530000 secs (94.34 Hz) >> (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> Solved 95 of 95 puzzles from top95.txt in 3.980000 secs (23.87 Hz) >> (221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> 648). >> Solved 11 of 11 puzzles from hardest.txt in 0.150000 secs (73.33 Hz) >> (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> [mevans@REDACTED ~]$ >> >> >> The initial implementation in Erlang: >> >> [mevans@REDACTED ~]$ ./sudoku >> ./sudoku:4: Warning: variable 'Args' is unused >> Solved 50 of 50 puzzles from easy50.txt in 0.492551 secs (101.51 Hz) >> (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). >> Solved 95 of 95 puzzles from top95.txt in 3.120420 secs (30.44 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> Solved 11 of 11 puzzles from hardest.txt in 0.147977 secs (74.34 Hz) >> (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). >> >> ________________________________________ >> From: Kresten Krab Thorup [krab@REDACTED] >> Sent: Sunday, March 27, 2011 9:28 AM >> To: Evans, Matthew >> Cc: Ahmed Omar; erlang-questions@REDACTED; Andreas Pauley >> Subject: Re: [erlang-questions 41] Re: A sudoku solver in Erlang compared >> to Python >> >> For reference, I tried to run this with Erjang and R14B01 (with BEAN and >> HiPE/o3) on my machine ... >> >> Oh, and I changed the square names to be atoms in stead of [X,Y] pairs, >> which speeds up things quite a lot too ... e.g. : >> >> squares() -> >> %% Returns a list of 81 square names, including "A1" etc. >> [erlang:list_to_atom([X,Y]) || X <- "ABCDEFGHI", Y <- "123456789"]. >> >> >> BEAM >> > sudoku_main:run_solver(["solve"]). >> Solved 50 of 50 puzzles from easy50.txt in 0.678921 secs (73.65 Hz) >> (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). >> Solved 95 of 95 puzzles from top95.txt in 4.373277 secs (21.72 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> Solved 11 of 11 puzzles from hardest.txt in 0.198780 secs (55.34 Hz) >> (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). >> ok >> >> Erjang >> > sudoku_main:run_solver(["solve"]). >> Solved 50 of 50 puzzles from easy50.txt in 0.653891 secs (76.47 Hz) >> (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). >> Solved 95 of 95 puzzles from top95.txt in 4.354102 secs (21.82 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> Solved 11 of 11 puzzles from hardest.txt in 0.208266 secs (52.82 Hz) >> (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). >> ok >> >> r14b01 / HiPE / o3 >> > sudoku_main:run_solver(["solve"]). >> Solved 50 of 50 puzzles from easy50.txt in 0.611918 secs (81.71 Hz) >> (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). >> Solved 95 of 95 puzzles from top95.txt in 3.759281 secs (25.27 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> Solved 11 of 11 puzzles from hardest.txt in 0.169039 secs (65.07 Hz) >> (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). >> ok >> >> This version does not use pmap (Erjang's scheduler still needs some work), >> but does use atoms for square names, and ct_expand. >> >> Cheers, >> >> Kresten >> >> >> On Mar 26, 2011, at 23:33 , Evans, Matthew wrote: >> >> > Awesome point, I was going to actually suggest that since this is all >> calculating static data he just pre-calculates it and puts it in a -define. >> Or even better, if he is brave he could write a parse transform. >> > >> > Using Ulf's ct_expand module ( >> http://forum.trapexit.org/viewtopic.php?p=20260) I get even better >> performance: >> > >> > Without the parse transform (but using native): >> > >> > 4> sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) >> > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> > ok >> > >> > >> > With a parse transform: >> > >> > 2> sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) >> > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> > >> > >> > The code looks like: >> > >> > -compile({parse_transform, ct_expand}). >> > >> > >> > squares() -> >> > %% Returns a list of 81 square names, including "A1" etc. >> > ct_expand:term( >> > [[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] >> > ). >> > col_squares() -> >> > %% All the square names for each column. >> > ct_expand:term( >> > [[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] >> > ). >> > row_squares() -> >> > %% All the square names for each row. >> > ct_expand:term( >> > [[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] >> > ). >> > box_squares() -> >> > %% All the square names for each box. >> > ct_expand:term( >> > [[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- >> ["123", "456", "789"]] >> > ). >> > >> > >> > ________________________________________ >> > From: Ahmed Omar [spawn.think@REDACTED] >> > Sent: Saturday, March 26, 2011 4:39 PM >> > To: Evans, Matthew >> > Cc: Andreas Pauley; erlang-questions@REDACTED >> > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang >> compared to Python >> > >> > BTW, beside looking into reducing eleminations (which requires deep >> dive), if u do some time profiling using eprof u will find that most of the >> time spent actually in the cross functions >> > sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 >> 35024918 [ 0.36] >> > >> > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED >> <mailto:spawn.think@REDACTED>> wrote: >> > actually instead of doing >> > >> > >> > >> > NonUniquePeers = shallow_flatten([S || S <- units(Square)]), >> > >> > >> > >> > PeerSet = sets:from_list(NonUniquePeers), >> > >> > >> > >> > PeersWithSelf = sets:to_list(PeerSet), >> > >> > >> > >> > >> > >> > >> > >> > >> > can't u just do >> > >> > >> > PeersWithSelf = lists:usort(NonUniquePeers). >> > >> > >> > >> > ? >> > >> > >> > >> > >> > >> > >> > >> > >> > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED >> <mailto:mevans@REDACTED>> wrote: >> > Good call....gb_sets should be faster. Compiled to native it runs faster >> still >> > >> > 9> sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >> > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> > ok >> > >> > ________________________________________ >> > From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] >> > Sent: Saturday, March 26, 2011 3:08 PM >> > To: Andreas Pauley >> > Cc: Evans, Matthew; erlang-questions@REDACTED<mailto: >> erlang-questions@REDACTED> >> > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang >> compared to Python >> > >> > using gb_sets instead of sets could decrease eliminations a bit and give >> u some boost. However, i didn't dive deeper into the code or the algorithm. >> > >> > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED >> <mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto: >> apauley@REDACTED>>> wrote: >> > Thanks, I've changed the compile options and this definitely makes it >> > somewhat faster: >> > >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> > >> > Strangely the native flag is not documented here: >> > http://www.erlang.org/doc/man/compile.html#file-2 >> > >> > The more interesting improvement would be to decrease the number of >> > eliminations performed, but for that I'll have to go deep into the >> > code :-) >> > >> > >> > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED >> <mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto: >> mevans@REDACTED>>> wrote: >> >> Without going deep into the code, one thing to try is compile with the >> native flag (running the tests from the VM shell here): >> >> >> >> Without native set: >> >> 18> c(sudoku). >> >> {ok,sudoku} >> >> 19> sudoku:print_results("top95.txt", "\n"). >> >> Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> >> ok >> >> >> >> >> >> With native set: >> >> 20> c(sudoku,[native]). >> >> {ok,sudoku} >> >> 21> sudoku:print_results("top95.txt", "\n"). >> >> Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> >> >> >> >> >> >> >> -----Original Message----- >> >> From: erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED><mailto: >> erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED>> [mailto: >> erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED><mailto: >> erlang-questions-bounces@REDACTED<mailto: >> erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley >> >> Sent: Friday, March 25, 2011 8:15 AM >> >> To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED >> ><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> >> Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> Python >> >> >> >> Hi all, >> >> >> >> In my quest to learn Erlang I've translated Peter Norvig's sudoku >> >> solver into Erlang: >> >> https://github.com/apauley/sudoku-in-erlang >> >> >> >> For comparison, my slightly modified version of Norvig's Python code >> >> can be found here: >> >> https://github.com/apauley/sudoku-by-norvig >> >> >> >> I like the pattern matching, it was fun to do the entire solution >> >> without a single if or case statement :-) >> >> Something that bothers me though is that the Python solution seems to >> >> be faster, even after I've made the Erlang solution use multiple >> >> processors. >> >> >> >> In order to compare the two solutions I counted the number of >> >> eliminations each performed. >> >> The eliminate function is the core of the solution, for example >> >> assigning a single value to a square is implemented as the elimination >> >> of all other values. >> >> >> >> With the Erlang implementation I get: >> >> sudoku-in-erlang$ ./sudoku >> >> All tests passed :-) >> >> Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> >> (93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> 1770). >> >> Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> 1797). >> >> Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> >> (32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> 1781). >> >> >> >> And with the Python implementation: >> >> sudoku-by-norvig$ ./sudoku.py >> >> All tests pass. >> >> Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> >> (33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> >> Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> >> (221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> 648). >> >> Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> >> (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> >> >> >> So according to the stats above, the Python solution performs less >> >> computations when given exactly the same input. >> >> The Erlang code is as close to the Python as I could make it, I've >> >> done more or less a direct translation of the algorithms used. >> >> >> >> I suspect that there are some lazy evaluation happening in the Python >> >> version, possibly generators, although I haven't pinpointed it yet. >> >> >> >> How can I improve my Erlang code in this solution? >> >> >> >> Kind regards, >> >> Andreas >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED >> ><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto: >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > -- >> > Best Regards, >> > - Ahmed Omar >> > http://nl.linkedin.com/in/adiaa >> > Follow me on twitter >> > @spawn_think<http://twitter.com/#!/spawn_think> >> > >> > >> > >> > >> > -- >> > Best Regards, >> > - Ahmed Omar >> > http://nl.linkedin.com/in/adiaa >> > Follow me on twitter >> > @spawn_think<http://twitter.com/#!/spawn_think> >> > >> > >> > >> > >> > -- >> > Best Regards, >> > - Ahmed Omar >> > http://nl.linkedin.com/in/adiaa >> > Follow me on twitter >> > @spawn_think<http://twitter.com/#!/spawn_think> >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think <http://twitter.com/#!/spawn_think> > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/dd5f06df/attachment.htm> From apauley@REDACTED Sun Mar 27 23:02:01 2011 From: apauley@REDACTED (Andreas Pauley) Date: Sun, 27 Mar 2011 23:02:01 +0200 Subject: [erlang-questions 58] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> Message-ID: <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> Thanks! I didn't think about the sets at all :-) https://github.com/apauley/sudoku-in-erlang/commit/a02fbcd001a9fbd876725008d58c52fcff9872d9 On Sat, Mar 26, 2011 at 9:08 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > using gb_sets instead of sets could decrease eliminations a bit and give u > some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED> wrote: >> >> Thanks, I've changed the compile options and this definitely makes it >> somewhat faster: >> >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> >> Strangely the native flag is not documented here: >> http://www.erlang.org/doc/man/compile.html#file-2 >> >> The more interesting improvement would be to decrease the number of >> eliminations performed, but for that I'll have to go deep into the >> code :-) >> >> >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> >> wrote: >> > Without going deep into the code, one thing to try is compile with the >> > native flag (running the tests from the VM shell here): >> > >> > Without native set: >> > 18> c(sudoku). >> > {ok,sudoku} >> > 19> ?sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > ok >> > >> > >> > With native set: >> > 20> c(sudoku,[native]). >> > {ok,sudoku} >> > 21> ?sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > >> > >> > >> > -----Original Message----- >> > From: erlang-questions-bounces@REDACTED >> > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas Pauley >> > Sent: Friday, March 25, 2011 8:15 AM >> > To: erlang-questions@REDACTED >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> > Python >> > >> > Hi all, >> > >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >> > solver into Erlang: >> > https://github.com/apauley/sudoku-in-erlang >> > >> > For comparison, my slightly modified version of Norvig's Python code >> > can be found here: >> > https://github.com/apauley/sudoku-by-norvig >> > >> > I like the pattern matching, it was fun to do the entire solution >> > without a single if or case statement :-) >> > Something that bothers me though is that the Python solution seems to >> > be faster, even after I've made the Erlang solution use multiple >> > processors. >> > >> > In order to compare the two solutions I counted the number of >> > eliminations each performed. >> > The eliminate function is the core of the solution, for example >> > assigning a single value to a square is implemented as the elimination >> > of all other values. >> > >> > With the Erlang implementation I get: >> > sudoku-in-erlang$ ./sudoku >> > All tests passed :-) >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> > 1770). >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> > 1781). >> > >> > And with the Python implementation: >> > sudoku-by-norvig$ ./sudoku.py >> > All tests pass. >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> > ?(33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> > 648). >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> > >> > So according to the stats above, the Python solution performs less >> > computations when given exactly the same input. >> > The Erlang code is as close to the Python as I could make it, I've >> > done more or less a direct translation of the algorithms used. >> > >> > I suspect that there are some lazy evaluation happening in the Python >> > version, possibly generators, although I haven't pinpointed it yet. >> > >> > How can I improve my Erlang code in this solution? >> > >> > Kind regards, >> > Andreas >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From spawn.think@REDACTED Sun Mar 27 23:21:01 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sun, 27 Mar 2011 23:21:01 +0200 Subject: [erlang-questions 59] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> Message-ID: <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> No problem, as said before u can also replace these lines : PeerSet = gb_sets:from_list(NonUniquePeers), PeersWithSelf = gb_sets:to_list(PeerSet), lists:delete(Square, PeersWithSelf). with just lists:delete(Square, lists:usort(NonUniquePeers)). (don't forget to check the rest of the thread too ;)) On Sun, Mar 27, 2011 at 11:02 PM, Andreas Pauley <apauley@REDACTED> wrote: > Thanks! > I didn't think about the sets at all :-) > > > https://github.com/apauley/sudoku-in-erlang/commit/a02fbcd001a9fbd876725008d58c52fcff9872d9 > > On Sat, Mar 26, 2011 at 9:08 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > > using gb_sets instead of sets could decrease eliminations a bit and give > u > > some boost. However, i didn't dive deeper into the code or the algorithm. > > > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED> > wrote: > >> > >> Thanks, I've changed the compile options and this definitely makes it > >> somewhat faster: > >> > >> > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > >> > >> Strangely the native flag is not documented here: > >> http://www.erlang.org/doc/man/compile.html#file-2 > >> > >> The more interesting improvement would be to decrease the number of > >> eliminations performed, but for that I'll have to go deep into the > >> code :-) > >> > >> > >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> > >> wrote: > >> > Without going deep into the code, one thing to try is compile with the > >> > native flag (running the tests from the VM shell here): > >> > > >> > Without native set: > >> > 18> c(sudoku). > >> > {ok,sudoku} > >> > 19> sudoku:print_results("top95.txt", "\n"). > >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >> > 1797). > >> > ok > >> > > >> > > >> > With native set: > >> > 20> c(sudoku,[native]). > >> > {ok,sudoku} > >> > 21> sudoku:print_results("top95.txt", "\n"). > >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >> > 1797). > >> > > >> > > >> > > >> > -----Original Message----- > >> > From: erlang-questions-bounces@REDACTED > >> > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas > Pauley > >> > Sent: Friday, March 25, 2011 8:15 AM > >> > To: erlang-questions@REDACTED > >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > >> > Python > >> > > >> > Hi all, > >> > > >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku > >> > solver into Erlang: > >> > https://github.com/apauley/sudoku-in-erlang > >> > > >> > For comparison, my slightly modified version of Norvig's Python code > >> > can be found here: > >> > https://github.com/apauley/sudoku-by-norvig > >> > > >> > I like the pattern matching, it was fun to do the entire solution > >> > without a single if or case statement :-) > >> > Something that bothers me though is that the Python solution seems to > >> > be faster, even after I've made the Erlang solution use multiple > >> > processors. > >> > > >> > In order to compare the two solutions I counted the number of > >> > eliminations each performed. > >> > The eliminate function is the core of the solution, for example > >> > assigning a single value to a square is implemented as the elimination > >> > of all other values. > >> > > >> > With the Erlang implementation I get: > >> > sudoku-in-erlang$ ./sudoku > >> > All tests passed :-) > >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > >> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > >> > 1770). > >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >> > 1797). > >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > >> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > >> > 1781). > >> > > >> > And with the Python implementation: > >> > sudoku-by-norvig$ ./sudoku.py > >> > All tests pass. > >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > >> > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > >> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > >> > 648). > >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > >> > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > >> > > >> > So according to the stats above, the Python solution performs less > >> > computations when given exactly the same input. > >> > The Erlang code is as close to the Python as I could make it, I've > >> > done more or less a direct translation of the algorithms used. > >> > > >> > I suspect that there are some lazy evaluation happening in the Python > >> > version, possibly generators, although I haven't pinpointed it yet. > >> > > >> > How can I improve my Erlang code in this solution? > >> > > >> > Kind regards, > >> > Andreas > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > Best Regards, > > - Ahmed Omar > > http://nl.linkedin.com/in/adiaa > > Follow me on twitter > > @spawn_think > > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/b4efbbad/attachment.htm> From spawn.think@REDACTED Sun Mar 27 23:23:28 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Sun, 27 Mar 2011 23:23:28 +0200 Subject: [erlang-questions 60] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> Message-ID: <AANLkTinipQgWi8F0LYFtz==n=f7-snG5BVArcC2+svz1@mail.gmail.com> running test after fixing the counter gives the following Solved 95 of 95 puzzles from top95.txt in 2.235227 secs (42.50 Hz) (210794 total eliminations, avg 2218.88, median 1536, max 10664, min 648). On Sun, Mar 27, 2011 at 11:21 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > No problem, as said before u can also replace these lines : > > PeerSet = gb_sets:from_list(NonUniquePeers), > PeersWithSelf = gb_sets:to_list(PeerSet), > lists:delete(Square, PeersWithSelf). > > with just > > lists:delete(Square, lists:usort(NonUniquePeers)). > > (don't forget to check the rest of the thread too ;)) > > On Sun, Mar 27, 2011 at 11:02 PM, Andreas Pauley <apauley@REDACTED>wrote: > >> Thanks! >> I didn't think about the sets at all :-) >> >> >> https://github.com/apauley/sudoku-in-erlang/commit/a02fbcd001a9fbd876725008d58c52fcff9872d9 >> >> On Sat, Mar 26, 2011 at 9:08 PM, Ahmed Omar <spawn.think@REDACTED> >> wrote: >> > using gb_sets instead of sets could decrease eliminations a bit and give >> u >> > some boost. However, i didn't dive deeper into the code or the >> algorithm. >> > >> > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED> >> wrote: >> >> >> >> Thanks, I've changed the compile options and this definitely makes it >> >> somewhat faster: >> >> >> >> >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> >> >> >> Strangely the native flag is not documented here: >> >> http://www.erlang.org/doc/man/compile.html#file-2 >> >> >> >> The more interesting improvement would be to decrease the number of >> >> eliminations performed, but for that I'll have to go deep into the >> >> code :-) >> >> >> >> >> >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> >> >> wrote: >> >> > Without going deep into the code, one thing to try is compile with >> the >> >> > native flag (running the tests from the VM shell here): >> >> > >> >> > Without native set: >> >> > 18> c(sudoku). >> >> > {ok,sudoku} >> >> > 19> sudoku:print_results("top95.txt", "\n"). >> >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> >> > 1797). >> >> > ok >> >> > >> >> > >> >> > With native set: >> >> > 20> c(sudoku,[native]). >> >> > {ok,sudoku} >> >> > 21> sudoku:print_results("top95.txt", "\n"). >> >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> >> > 1797). >> >> > >> >> > >> >> > >> >> > -----Original Message----- >> >> > From: erlang-questions-bounces@REDACTED >> >> > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas >> Pauley >> >> > Sent: Friday, March 25, 2011 8:15 AM >> >> > To: erlang-questions@REDACTED >> >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> >> > Python >> >> > >> >> > Hi all, >> >> > >> >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >> >> > solver into Erlang: >> >> > https://github.com/apauley/sudoku-in-erlang >> >> > >> >> > For comparison, my slightly modified version of Norvig's Python code >> >> > can be found here: >> >> > https://github.com/apauley/sudoku-by-norvig >> >> > >> >> > I like the pattern matching, it was fun to do the entire solution >> >> > without a single if or case statement :-) >> >> > Something that bothers me though is that the Python solution seems to >> >> > be faster, even after I've made the Erlang solution use multiple >> >> > processors. >> >> > >> >> > In order to compare the two solutions I counted the number of >> >> > eliminations each performed. >> >> > The eliminate function is the core of the solution, for example >> >> > assigning a single value to a square is implemented as the >> elimination >> >> > of all other values. >> >> > >> >> > With the Erlang implementation I get: >> >> > sudoku-in-erlang$ ./sudoku >> >> > All tests passed :-) >> >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> >> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> >> > 1770). >> >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> >> > 1797). >> >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> >> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> >> > 1781). >> >> > >> >> > And with the Python implementation: >> >> > sudoku-by-norvig$ ./sudoku.py >> >> > All tests pass. >> >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> >> > (33059 total eliminations, avg 661.00, median 648, max 830, min >> 648). >> >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> >> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> >> > 648). >> >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> >> > (9436 total eliminations, avg 857.00, median 817, max 1198, min >> 648). >> >> > >> >> > So according to the stats above, the Python solution performs less >> >> > computations when given exactly the same input. >> >> > The Erlang code is as close to the Python as I could make it, I've >> >> > done more or less a direct translation of the algorithms used. >> >> > >> >> > I suspect that there are some lazy evaluation happening in the Python >> >> > version, possibly generators, although I haven't pinpointed it yet. >> >> > >> >> > How can I improve my Erlang code in this solution? >> >> > >> >> > Kind regards, >> >> > Andreas >> >> > _______________________________________________ >> >> > erlang-questions mailing list >> >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> > >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > -- >> > Best Regards, >> > - Ahmed Omar >> > http://nl.linkedin.com/in/adiaa >> > Follow me on twitter >> > @spawn_think >> > >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think <http://twitter.com/#!/spawn_think> > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/94fc85ef/attachment.htm> From apauley@REDACTED Sun Mar 27 23:36:57 2011 From: apauley@REDACTED (Andreas Pauley) Date: Sun, 27 Mar 2011 23:36:57 +0200 Subject: [erlang-questions 61] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> Message-ID: <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> On Sun, Mar 27, 2011 at 11:21 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > No problem, as said before u can also replace these lines : > PeerSet = gb_sets:from_list(NonUniquePeers), > PeersWithSelf = gb_sets:to_list(PeerSet), > lists:delete(Square, PeersWithSelf). > with just > lists:delete(Square,?lists:usort(NonUniquePeers)). Thanks, done: https://github.com/apauley/sudoku-in-erlang/commit/1852c68baf525524d21ea508dead3f51820141b5 > (don't forget to check the rest of the thread too ;)) Yes, thanks everyone, all the suggestions are awesome, I'm going through them one by one :-) From apauley@REDACTED Sun Mar 27 23:57:17 2011 From: apauley@REDACTED (Andreas Pauley) Date: Sun, 27 Mar 2011 23:57:17 +0200 Subject: [erlang-questions 62] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB26@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB26@vvexch.verivue.com> Message-ID: <AANLkTikdZ-b9o9YtR7x7DmTD4A9aF_zhH3sJrzb2K-mw@mail.gmail.com> On Sat, Mar 26, 2011 at 9:46 PM, Evans, Matthew <mevans@REDACTED> wrote: > Cool... > > Might also want to look at what happens if all the other Erlang modules (esp. lists, sets and dict for your app) are made native: Ok, so to try this I copied lists.erl etc. into my own source dir, and then I compiled it to beams using the native flag. Is that what you meant? Because that will be cool for a once-off experiment, but not so cool in the long run. Doing that means I'm taking responsibility for doing it every time I upgrade my Erlang version etc. Is there a way to do it with less maintenance? From apauley@REDACTED Mon Mar 28 00:03:58 2011 From: apauley@REDACTED (Andreas Pauley) Date: Mon, 28 Mar 2011 00:03:58 +0200 Subject: [erlang-questions 63] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTinY5HuOS5cCjqxPUFtn0Zuhq0DnC7MWpxyEiAeP@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTinY5HuOS5cCjqxPUFtn0Zuhq0DnC7MWpxyEiAeP@mail.gmail.com> Message-ID: <AANLkTikcXMf71KFMLFcc_BY8J-RgVT7qoQLoZfhvW_nb@mail.gmail.com> Hi Kenny, These are the changes I made to use multiple processors: https://github.com/apauley/sudoku-in-erlang/commit/306333757e3df2fcafdb94604b8963dd2b7c79e0 https://github.com/apauley/sudoku-in-erlang/commit/58a5ac65b83219f85b34bb955e75c7e0db6f5122 You're right, not much changed :-) Feel free to show me if there is a cleaner way to do this. Regards, Andreas On Sat, Mar 26, 2011 at 10:21 PM, Kenny Stone <kennethstone@REDACTED> wrote: > Are you solving the puzzles in parallel? ?It might be revealing to see the > implementation differences ?in python and erlang for solving a lot of > puzzles when the code parallelized. ?The erlang code won't change much. > > On Sat, Mar 26, 2011 at 3:14 PM, Evans, Matthew <mevans@REDACTED> wrote: >> >> Good call....gb_sets should be faster. Compiled to native it runs faster >> still >> >> 9> sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> 1792). >> ok >> >> ________________________________________ >> From: Ahmed Omar [spawn.think@REDACTED] >> Sent: Saturday, March 26, 2011 3:08 PM >> To: Andreas Pauley >> Cc: Evans, Matthew; erlang-questions@REDACTED >> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared >> to Python >> >> using gb_sets instead of sets could decrease eliminations a bit and give u >> some boost. However, i didn't dive deeper into the code or the algorithm. >> >> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley >> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: >> Thanks, I've changed the compile options and this definitely makes it >> somewhat faster: >> >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> >> Strangely the native flag is not documented here: >> http://www.erlang.org/doc/man/compile.html#file-2 >> >> The more interesting improvement would be to decrease the number of >> eliminations performed, but for that I'll have to go deep into the >> code :-) >> >> >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew >> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: >> > Without going deep into the code, one thing to try is compile with the >> > native flag (running the tests from the VM shell here): >> > >> > Without native set: >> > 18> c(sudoku). >> > {ok,sudoku} >> > 19> ?sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > ok >> > >> > >> > With native set: >> > 20> c(sudoku,[native]). >> > {ok,sudoku} >> > 21> ?sudoku:print_results("top95.txt", "\n"). >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > >> > >> > >> > -----Original Message----- >> > From: >> > erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED> >> > [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>] >> > On Behalf Of Andreas Pauley >> > Sent: Friday, March 25, 2011 8:15 AM >> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> > Python >> > >> > Hi all, >> > >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >> > solver into Erlang: >> > https://github.com/apauley/sudoku-in-erlang >> > >> > For comparison, my slightly modified version of Norvig's Python code >> > can be found here: >> > https://github.com/apauley/sudoku-by-norvig >> > >> > I like the pattern matching, it was fun to do the entire solution >> > without a single if or case statement :-) >> > Something that bothers me though is that the Python solution seems to >> > be faster, even after I've made the Erlang solution use multiple >> > processors. >> > >> > In order to compare the two solutions I counted the number of >> > eliminations each performed. >> > The eliminate function is the core of the solution, for example >> > assigning a single value to a square is implemented as the elimination >> > of all other values. >> > >> > With the Erlang implementation I get: >> > sudoku-in-erlang$ ./sudoku >> > All tests passed :-) >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> > 1770). >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, min >> > 1797). >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> > 1781). >> > >> > And with the Python implementation: >> > sudoku-by-norvig$ ./sudoku.py >> > All tests pass. >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> > ?(33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> > 648). >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> > >> > So according to the stats above, the Python solution performs less >> > computations when given exactly the same input. >> > The Erlang code is as close to the Python as I could make it, I've >> > done more or less a direct translation of the algorithms used. >> > >> > I suspect that there are some lazy evaluation happening in the Python >> > version, possibly generators, although I haven't pinpointed it yet. >> > >> > How can I improve my Erlang code in this solution? >> > >> > Kind regards, >> > Andreas >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think<http://twitter.com/#!/spawn_think> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From mevans@REDACTED Mon Mar 28 00:13:04 2011 From: mevans@REDACTED (Evans, Matthew) Date: Sun, 27 Mar 2011 18:13:04 -0400 Subject: [erlang-questions 64] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTikdZ-b9o9YtR7x7DmTD4A9aF_zhH3sJrzb2K-mw@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB26@vvexch.verivue.com> <AANLkTikdZ-b9o9YtR7x7DmTD4A9aF_zhH3sJrzb2K-mw@mail.gmail.com> Message-ID: <3E09680E-8CAB-44B1-86BB-BD78BBDA73AB@verivue.com> Apparently you can use the option --enable-native-libs with configure when you build Erlang. Sent from my iPhone On Mar 27, 2011, at 5:57 PM, "Andreas Pauley" <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: On Sat, Mar 26, 2011 at 9:46 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: Cool... Might also want to look at what happens if all the other Erlang modules (esp. lists, sets and dict for your app) are made native: Ok, so to try this I copied lists.erl etc. into my own source dir, and then I compiled it to beams using the native flag. Is that what you meant? Because that will be cool for a once-off experiment, but not so cool in the long run. Doing that means I'm taking responsibility for doing it every time I upgrade my Erlang version etc. Is there a way to do it with less maintenance? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/e4b36441/attachment.htm> From mojitotech@REDACTED Mon Mar 28 00:55:31 2011 From: mojitotech@REDACTED (Mojito Sorbet) Date: Sun, 27 Mar 2011 18:55:31 -0400 Subject: [erlang-questions 65] Erlang/OTP configure can not find wxWidgets In-Reply-To: <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> Message-ID: <4D8FC063.3030203@gmail.com> I am installing the latest Erlang/OTP R14B02 on a new Ubuntu 10.10 system. I have installed all the dependencies, including wxWidgets, which I need. But when I run ./configure, it says it can not find the wx/stc/stc.h file, and so can not link the wx interface. I have all the wx components installed that ought to include that, so I went looking for the file. It is there, in /include. But ./configure is looking for it somewhere else. How do I convince it to look in the right place? I installed wxWidgets using the Ubuntu package tool, which runs apt-get behind the scenes. I had this working on Ubuntu 9.04 with a previous Erlang release, but do not remember how I did it. From andrew@REDACTED Mon Mar 28 01:18:25 2011 From: andrew@REDACTED (Andrew Thompson) Date: Sun, 27 Mar 2011 19:18:25 -0400 Subject: [erlang-questions 66] Re: HiPE or native....good results, but confused... In-Reply-To: <4D8F6F90.8000003@cs.ntua.gr> References: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB27@vvexch.verivue.com> <AANLkTikc5h5j_YDBiBP5XAB9bZAk3OZ9ufcyq2hoBME4@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2C@vvexch.verivue.com> <20110327165813.GL20461@hijacked.us> <4D8F6F90.8000003@cs.ntua.gr> Message-ID: <20110327231825.GM20461@hijacked.us> On Sun, Mar 27, 2011 at 08:10:40PM +0300, Kostis Sagonas wrote: > You are of course free to do whatever you want, but wouldn't send a > bug report (and a test case for) these problems be a better approach > than sending just this mail to the mailing list? > Yes (and I have done so in the past). I'll play with it again and see if I can come up with some more bug reports. I'm just noting that I've always had *some* issue with HiPE, although I will admit that its never the same issue twice (which means bugs are being fixed). Andrew From kenrobinsonster@REDACTED Mon Mar 28 02:32:25 2011 From: kenrobinsonster@REDACTED (Ken Robinson) Date: Mon, 28 Mar 2011 10:32:25 +1000 Subject: [erlang-questions 67] Problem using -(extends) with gen_server Message-ID: <AANLkTi=EBK5fFsSCPnWMf8eZYhm671grf+XNrs_wV7W4@mail.gmail.com> Hi All, I was trying to use -(extends) with gen_server. All the callback methods remained the same. The only exported function from the extended gen_server was the handle_call/3. What I found was it seemed to be failing at the init stage. Does gen_server work with -(extends) or should it be avoided. Does -(extends) only work with vanilla modules? -- regards, Ken Robinson Mob +61438681120 Home +61738523767 From erlang@REDACTED Mon Mar 28 06:09:42 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 27 Mar 2011 21:09:42 -0700 Subject: [erlang-questions 68] starting and managing a resident erlang server on mac (or other OSs) Message-ID: <AANLkTi=YhCcY-3FDqy6sLAD7nJ0Szbm38Hg9SP43D6Ru@mail.gmail.com> I want to run a resident distributed erlang node on my macbook. It should be started when I reboot my machine. It should be restarted if it crashes. Nothing nasty should happen when I open/close the lid :-) (I guess I also need to "manage" the node occasional - so I'd need some remote control facilities) What is "best practice" for doing this - does anybody maintain scripts of this nature? (aside it strikes me that this cannot be an uncommon problem - since we are now in uncharted "OS specific" territory it would be nice to know the answer to this question for all the other "popular" OSs - any tips ?) /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110327/80587a2b/attachment.htm> From boris.muehmer@REDACTED Mon Mar 28 06:29:51 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 28 Mar 2011 06:29:51 +0200 Subject: [erlang-questions 69] Re: Erlang/OTP configure can not find wxWidgets In-Reply-To: <4D8FC063.3030203@gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> <4D8FC063.3030203@gmail.com> Message-ID: <AANLkTi=E9J2uXEANZoNRvZBC+nxJE1=i_2_YeahNZDQW@mail.gmail.com> I took me some time to really get all the dependencies. Maybe have a look at my "mental note" for installing Erlang/OTP from source on Ubuntu systems at for comparison: http://boris.muehmer.de/2010/10/16/installing-erlangotp-r14b-on-ubuntu-from-source/ Hopefully this helps... - boris 2011/3/28 Mojito Sorbet <mojitotech@REDACTED>: > I am installing the latest Erlang/OTP R14B02 on a new Ubuntu 10.10 system. I > have installed all the dependencies, including wxWidgets, which I need. > > But when I run ./configure, it says it can not find the wx/stc/stc.h file, > and so can not link the wx interface. > > I have all the wx components installed that ought to include that, so I went > looking for the file. ?It is there, in /include. ?But ./configure is looking > for it somewhere else. ?How do I convince it to look in the right place? > > I installed wxWidgets using the Ubuntu package tool, which runs apt-get > behind the scenes. > > I had this working on Ubuntu 9.04 with a previous Erlang release, but do not > remember how I did it. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From rapsey@REDACTED Mon Mar 28 06:41:59 2011 From: rapsey@REDACTED (Rapsey) Date: Mon, 28 Mar 2011 06:41:59 +0200 Subject: [erlang-questions 70] Re: starting and managing a resident erlang server on mac (or other OSs) In-Reply-To: <AANLkTi=YhCcY-3FDqy6sLAD7nJ0Szbm38Hg9SP43D6Ru@mail.gmail.com> References: <AANLkTi=YhCcY-3FDqy6sLAD7nJ0Szbm38Hg9SP43D6Ru@mail.gmail.com> Message-ID: <BANLkTimj0=CLHaj-HD4jTJMp-W4wF-QWhw@mail.gmail.com> Mac actually has a pretty nice way to run programs as daemons. Similar to upstart on linux, but better. You need a .plist xml. Something like this: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" " http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>NAME_OF_YOUR_PROGRAM</string> <key>EnvironmentVariables</key> <dict> <key>HOME</key> <string>~</string> </dict> <key>OnDemand</key> <false/> <key>ProgramArguments</key> <array> <string>EXEC_WORD1</string> <string>EXEC_WORD2</string> <string>EXEC_WORDX</string> </array> <key>RunAtLoad</key> <true/> <key>KeepAlive</key> <true/> <key>UserName</key> <string>NAME_OF_USER_TO_RUN_AS</string> <key>Group</key> <string>NAME_OF_GROUP_TO_RUN_AS</string> </dict> </plist> For every word in your execution line, you need a separate <string> in ProgramArguments. Add it to /Library/LaunchDaemons/yourxml.plist If it's there, it will be executed on startup. To load (run): sudo launchctl load /Library/LaunchDaemons/yourxml.plist To unload (stop): sudo launchctl unload /Library/LaunchDaemons/yourxml.plist I usually put the erl execution in a bash script and execute the script in the plist. #!/bin/sh cd /Path/To/My/Ebin erl +Bd +P 1000000 -env ERL_MAX_PORTS 100000 -noinput +K true +A 32 -mnesia dir '"/Path/To/My/priv/mnesia"' -eval "application:start(MY_APP,permanent)" You can even see what your program is printing to stdout. If you run /Applications/Utilities/Console -> Console Messages http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPSystemStartup/Articles/LaunchOnDemandDaemons.html Sergej On Mon, Mar 28, 2011 at 6:09 AM, Joe Armstrong <erlang@REDACTED> wrote: > I want to run a resident distributed erlang node on my macbook. > It should be started when I reboot my machine. > It should be restarted if it crashes. Nothing nasty should happen when I > open/close > the lid :-) (I guess I also need to "manage" the node occasional - so I'd > need some > remote control facilities) > > What is "best practice" for doing this - does anybody maintain scripts of > this nature? > > (aside it strikes me that this cannot be an uncommon problem - since we are > now in > uncharted "OS specific" territory it would be nice to know the answer to > this question > for all the other "popular" OSs - any tips ?) > > /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/b765bbf6/attachment.htm> From sam@REDACTED Mon Mar 28 08:07:12 2011 From: sam@REDACTED (Sam Bobroff) Date: Mon, 28 Mar 2011 17:07:12 +1100 Subject: [erlang-questions 71] Erlang release generation Message-ID: <4D902590.8060409@m5net.com> Hi Erlangers, I've been trying to work out how to "properly" build Erlang releases (on a Linux system, to be deployed on another Linux system), and I'm getting nowhere fast so I'm hoping someone here will have some useful pointers or hints to help me out. I'm using Erlang R14B, but I assume the problems I'm having are to do with understanding, rather than bugs so the version probably doesn't matter much ;-) For context, I've read a lot of the online Erlang documentation including the sections on systools, reltool, sys, rel (and I've read the Rebar documentation as well, I'll get to that later) and I'm familiar with setting up OTP applications but this issue is confusing me more and more: there doesn't seem to be any kind of overview that tells me how the various pieces fit together or what I'm supposed to do. Is there some standard document I'm missing? "All" I want to do is go from a set of Erlang source, representing one or more applications (presumably in a set of src and ebin dirs), to an installed Erlang system that will start those applications. (Which I'm obviously intending to package up for distribution but that's a separate issue.) I realize I'm rambling so I'll try to phrase some specific questions: Is reltool:create_target() the correct way to build a release? If so, how do I get reltool:create_target() to include my apps? I've tried using lib_dirs and -pa in various combinations but I always get "Release NAME uses non existing application APP". I haven't yet tried installing my application directly to the system Erlang (/usr/lib/erlang/lib in my case) because the reltool doc clearly states that "Applications may also be located outside root_dir ...". Does this just not work? I'm not keen on installing my apps into the system Erlang (as root) just so that I can build a release! I've discovered that if I use reltool:create_target() to build a release (and don't include any custom apps, so it works), I can't then use that release as a development environment because running reltools fails on all applications, even kernel. It seems to be that reltool doesn't understand that things have been packaged up as .ez files because manually extracting them all allows it to succeed. Is there a way to get reltool to build a release that isn't zipped up, or to understand the zipped files? Should I instead be following the "target_system" module that's in the "Creating a First Target System" section of the documentation? It seems pretty odd to cut and paste a big chunk of example code out of the docs to do a basic task... and it's also very slow because of the way it tars everything up just to untar it again. This is, however, the only way I've managed to get a release of my own app actually built. Should I just build .rel files by hand and basically re-implement target_system.erl (but more the way I want it to work)? I've also tried using rebar, but I'm getting the same problem: I can't get the created release to include any of my own applications. Here's exactly what I tried: mkdir rebar-tmp cd rebar-tmp wget http://hg.basho.com/rebar/downloads/rebar chmod +x rebar mkdir mysample cd mysample ../rebar create-app appid=mysample ../rebar compile cd .. mkdir rel cd rel ../rebar create-node nodeid=mysample cd .. echo '{sub_dirs, ["rel"]}.' > rebar.config ./rebar generate (Runs successfully, but no mysample is included, as expected.) edit rel/reltool.config and add "{lib_dirs, ["../../"]} to the top sys list, ", mysample" to the mysample rel tuple and ",{app, mysample, [{incl_cond, include}]}" to the top sys list (exactly following the rebar documentation). ./rebar generate And I get: ==> rel (generate) {"init terminating in do_boot","Release mysample uses non existing application mysample"} Crash dump was written to: erl_crash.dump init terminating in do_boot (Release mysample uses non existing application mysample) What am I doing wrong? :-) Thanks for reading all the way :-) Peace, Sam. From max.lapshin@REDACTED Mon Mar 28 08:52:08 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 28 Mar 2011 10:52:08 +0400 Subject: [erlang-questions 72] Re: code:lib_dir and .ez archives In-Reply-To: <1301248066671-3409584.post@n4.nabble.com> References: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> <1301248066671-3409584.post@n4.nabble.com> Message-ID: <AANLkTimhcFEq4+A25+1iRFXvR9dendnDGasbWBBLnzFL@mail.gmail.com> There is other problem with escript: it cannot go background. It would be very convenient for me to distribute erlyvideo in small single-file launch scripts, but I can't do it because it doesn't allow to go background. From ingela.andin@REDACTED Mon Mar 28 08:58:55 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 28 Mar 2011 08:58:55 +0200 Subject: [erlang-questions 73] Re: dialyzer and ssl In-Reply-To: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> References: <AANLkTin6=qr4Nn7=eo1yfzKZJP2v8jVmz=a7JPhF2zVr@mail.gmail.com> Message-ID: <AANLkTinUj81iTtvkai4BvU+Pivh9Oj6rxR0M9kVROVMN@mail.gmail.com> Hi! 2011/3/24 Roberto Ostinelli <roberto@REDACTED>: > dear list, > > i'm getting this dialyzer warning i cannot understand: > > "The pattern 'ok' can never match the type {'error',_} | > {'ok',{'sslsocket',_,_}}" > > this points to line 4 of this code extract: > > 1. case ssl:transport_accept(ListenSocket) of > 2. ?? {ok, {sslsocket, _, _} = Sock} -> You should not match a record as its tuple representation, it breaks the abstraction and possible your code if we add a record field, and in this case the record is not documented either, the "sslsocket" is documented to be opaque to the user so you ought to match only {ok, Socket} > 3. ?? ??? case ssl:ssl_accept(Sock, 60000) of > 4. ?? ??? ??? ok -> > ??? ??? ??? ??? ... > 5. ?? ??? ??? {error, _Reason} -> > ??? ??? ??? ??? ... > 6. ?? ??? end > 7. ?? {error, _Reason} -> > ??? ??? ... > 8. end > > afaik, ssl_accept/2 returns ok | {error, Reason} as per the specs: > http://www.erlang.org/doc/man/ssl.html#ssl_accept-2 > > is this due to some discrepancy in docs/specs? or am i missing something? As for ssl:ssl_accept/2 it can return both ok and {ok, Socket} the latter will be returned if you upgrade a tcp-socket to an ssl-socket and ok will be returned in the normal case to retain backwards compatibility. (You already have access to the sslsocket from the transport_accept in this case). This is documented even in r13. We wanted to return {ok, Socket} in both cases but we had to priorities the backwards compatibility so not using the latest version could very well result in discrepancies, but I have not looked deeper to verify what is your actual problem. Regards Ingela Erlang/OTP team - Ericsson AB From sedrik@REDACTED Mon Mar 28 09:07:02 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Mon, 28 Mar 2011 09:07:02 +0200 Subject: [erlang-questions 74] Re: Erlang/OTP configure can not find wxWidgets In-Reply-To: <AANLkTi=E9J2uXEANZoNRvZBC+nxJE1=i_2_YeahNZDQW@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> <4D8FC063.3030203@gmail.com> <AANLkTi=E9J2uXEANZoNRvZBC+nxJE1=i_2_YeahNZDQW@mail.gmail.com> Message-ID: <AANLkTi==5Cm1BSprMBKjvTdC1TFneOfQzt_zspX9qC1c@mail.gmail.com> I have recently been working with the compile stages of the Erlang environment and found that you need to install the following packages for wx. libwxbase-2.8-dev libgl1-mesa-dev libglu1-mesa-dev libwxgtk2.8-dev libglut3-devI alos found an archived email somewhere stating that the following packages should do for wx support (I have not tested it yet and lost the source) freeglut3-dev libwxgtk2.8-dev g++ On Mon, Mar 28, 2011 at 6:29 AM, Boris M?hmer <boris.muehmer@REDACTED>wrote: > I took me some time to really get all the dependencies. > > Maybe have a look at my "mental note" for installing Erlang/OTP from > source on Ubuntu systems at for comparison: > > > http://boris.muehmer.de/2010/10/16/installing-erlangotp-r14b-on-ubuntu-from-source/ > > Hopefully this helps... > > - boris > > 2011/3/28 Mojito Sorbet <mojitotech@REDACTED>: > > I am installing the latest Erlang/OTP R14B02 on a new Ubuntu 10.10 > system. I > > have installed all the dependencies, including wxWidgets, which I need. > > > > But when I run ./configure, it says it can not find the wx/stc/stc.h > file, > > and so can not link the wx interface. > > > > I have all the wx components installed that ought to include that, so I > went > > looking for the file. It is there, in /include. But ./configure is > looking > > for it somewhere else. How do I convince it to look in the right place? > > > > I installed wxWidgets using the Ubuntu package tool, which runs apt-get > > behind the scenes. > > > > I had this working on Ubuntu 9.04 with a previous Erlang release, but do > not > > remember how I did it. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/dccf93a8/attachment.htm> From raimo+erlang-questions@REDACTED Mon Mar 28 09:34:03 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 28 Mar 2011 09:34:03 +0200 Subject: [erlang-questions 75] Re: New Mailing list software In-Reply-To: <56F659B0-8538-47C2-85EA-7DA768050683@erlang-solutions.com> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <56F659B0-8538-47C2-85EA-7DA768050683@erlang-solutions.com> Message-ID: <20110328073403.GA13069@erix.ericsson.se> On Fri, Mar 25, 2011 at 02:22:50PM -0700, Fr?d?ric Trottier-H?bert wrote: > No junk for me, although I worry about the XSS issue pointed earlier. Sorry, which XSS issue? > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-03-25, at 14:18 PM, Koener Antoine wrote: > > > Hi, > > > > Every single mail seems to be seen as Junk now in Apple Mail. > > > > Is this just me ? :) > > > > > > PS: I know that there's many other imap clients out there :-) > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Mon Mar 28 09:35:51 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 28 Mar 2011 09:35:51 +0200 Subject: [erlang-questions 76] Re: New Mailing list software In-Reply-To: <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <4D8D0A5E.3000208@telegraphics.com.au> <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> Message-ID: <20110328073551.GB13069@erix.ericsson.se> On Fri, Mar 25, 2011 at 02:44:54PM -0700, Simon Thompson wrote: > Something peculiar happening with header padding for me: getting ints in headers so my automated filing is not spotting them ... (this filing done before hitting client) Yes. It is auto numbering of posts. I thought it would be something to facilitate threading in the clients, but seems to cause confusion. I think I will shut it off. > > Simon > > Sent from my iPhone > > On 25 Mar 2011, at 14:34, Toby Thain <toby@REDACTED> wrote: > > > On 25/03/11 5:18 PM, Koener Antoine wrote: > >> Hi, > >> > >> Every single mail seems to be seen as Junk now in Apple Mail. > >> > >> Is this just me ? :) > > > > Yes, it's just you; that is a client side function. > > > > --T > > > > > >> > >> > >> PS: I know that there's many other imap clients out there :-) > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From mjtruog@REDACTED Mon Mar 28 09:50:32 2011 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 28 Mar 2011 00:50:32 -0700 Subject: [erlang-questions 77] Re: starting and managing a resident erlang server on mac (or other OSs) In-Reply-To: <BANLkTimj0=CLHaj-HD4jTJMp-W4wF-QWhw@mail.gmail.com> References: <AANLkTi=YhCcY-3FDqy6sLAD7nJ0Szbm38Hg9SP43D6Ru@mail.gmail.com> <BANLkTimj0=CLHaj-HD4jTJMp-W4wF-QWhw@mail.gmail.com> Message-ID: <4D903DC8.3070407@gmail.com> The rebar release includes a script that should be suitable for running the Erlang system as a daemon on OSX. The command sequence is here: https://bitbucket.org/basho/rebar/wiki/ReleaseHandling The configuration files I currently like are here (which are largely defaults): https://github.com/okeuday/CloudI/blob/master/src/reltool.config.in (normal name is reltool.config) https://github.com/okeuday/CloudI/blob/master/src/rebar.config On 03/27/2011 09:41 PM, Rapsey wrote: > Mac actually has a pretty nice way to run programs as daemons. Similar to upstart on linux, but better. > You need a .plist xml. Something like this: > > <?xml version="1.0" encoding="UTF-8"?> > <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> > <plist version="1.0"> > <dict> > <key>Label</key> > <string>NAME_OF_YOUR_PROGRAM</string> > <key>EnvironmentVariables</key> > <dict> > <key>HOME</key> > <string>~</string> > </dict> > <key>OnDemand</key> > <false/> > <key>ProgramArguments</key> > <array> > <string>EXEC_WORD1</string> > <string>EXEC_WORD2</string> > <string>EXEC_WORDX</string> > </array> > <key>RunAtLoad</key> > <true/> > <key>KeepAlive</key> > <true/> > <key>UserName</key> > <string>NAME_OF_USER_TO_RUN_AS</string> > <key>Group</key> > <string>NAME_OF_GROUP_TO_RUN_AS</string> > </dict> > </plist> > > For every word in your execution line, you need a separate <string> in ProgramArguments. > Add it to /Library/LaunchDaemons/yourxml.plist > If it's there, it will be executed on startup. > > To load (run): sudo launchctl load /Library/LaunchDaemons/yourxml.plist > To unload (stop): sudo launchctl unload /Library/LaunchDaemons/yourxml.plist > > I usually put the erl execution in a bash script and execute the script in the plist. > > #!/bin/sh > cd /Path/To/My/Ebin > erl +Bd +P 1000000 -env ERL_MAX_PORTS 100000 -noinput +K true +A 32 -mnesia dir '"/Path/To/My/priv/mnesia"' -eval "application:start(MY_APP,permanent)" > > You can even see what your program is printing to stdout. If you run /Applications/Utilities/Console -> Console Messages > > http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPSystemStartup/Articles/LaunchOnDemandDaemons.html > > > Sergej > > On Mon, Mar 28, 2011 at 6:09 AM, Joe Armstrong <erlang@REDACTED <mailto:erlang@REDACTED>> wrote: > > I want to run a resident distributed erlang node on my macbook. > It should be started when I reboot my machine. > It should be restarted if it crashes. Nothing nasty should happen when I open/close > the lid :-) (I guess I also need to "manage" the node occasional - so I'd need some > remote control facilities) > > What is "best practice" for doing this - does anybody maintain scripts of this nature? > > (aside it strikes me that this cannot be an uncommon problem - since we are now in > uncharted "OS specific" territory it would be nice to know the answer to this question > for all the other "popular" OSs - any tips ?) > > /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED <mailto:erlang-questions@REDACTED> > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/dcd42c3f/attachment.htm> From tobias.schlager@REDACTED Mon Mar 28 10:23:07 2011 From: tobias.schlager@REDACTED (Tobias Schlager) Date: Mon, 28 Mar 2011 10:23:07 +0200 Subject: [erlang-questions 78] [Announcement] new Maven Erlang Plug-In 2.0.0 considered stable Message-ID: <4D90456B.7010306@lindenbaum.eu> Hi, there's a new attempt of integrating erlang with maven. http://erlang-plugin.sourceforge.net/ Perhaps it's an option for those of you already using maven for non-erlang projects. It's avaliable directly from the 'central' repository. Just search for 'maven-erlang-plugin'. Regards Tobias Schlager From apauley@REDACTED Mon Mar 28 11:59:17 2011 From: apauley@REDACTED (Andreas Pauley) Date: Mon, 28 Mar 2011 11:59:17 +0200 Subject: [erlang-questions 79] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> Message-ID: <AANLkTint5mO01SkWhrrsGK9FuJ5z-qxBGMzWc-HKJNmw@mail.gmail.com> Wow, ct_expand made quite a big difference: https://github.com/apauley/sudoku-in-erlang/commit/10cb1a106d6a1681155b64b158a21bfcf9b0488e Solved 50 of 50 puzzles from easy50.txt in 1.277456 secs (39.14 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 8.024644 secs (11.84 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.384212 secs (28.63 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). Previously: Solved 50 of 50 puzzles from easy50.txt in 1.960654 secs (25.50 Hz) (92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). Solved 95 of 95 puzzles from top95.txt in 13.472294 secs (7.05 Hz) (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). Solved 11 of 11 puzzles from hardest.txt in 0.600741 secs (18.31 Hz) (33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). I don't see the same speeds as other people, but I guess it's because my Macbook only has 2 cores. On Sun, Mar 27, 2011 at 12:33 AM, Evans, Matthew <mevans@REDACTED> wrote: > Awesome point, I was going to actually suggest that since this is all calculating static data he just pre-calculates it and puts it in a -define. Or even better, if he is brave he could write a parse transform. > > Using Ulf's ct_expand module (http://forum.trapexit.org/viewtopic.php?p=20260) I get even better performance: > > Without the parse transform (but using native): > > 4> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.827966 secs (114.74 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > > With a parse transform: > > 2> ?sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.469908 secs (202.17 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > > > The code looks like: > > -compile({parse_transform, ct_expand}). > > > squares() -> > ? ?%% Returns a list of 81 square names, including "A1" etc. > ? ?ct_expand:term( > ? ? ? ?[[X,Y] || X <- "ABCDEFGHI", Y <- "123456789"] > ? ?). > col_squares() -> > ? ?%% All the square names for each column. > ? ?ct_expand:term( > ? ? ? ?[[[X,Y] || X <- "ABCDEFGHI", Y <- [C]] || C <- "123456789"] > ? ? ?). > row_squares() -> > ? ?%% All the square names for each row. > ? ?ct_expand:term( > ? ? ? ?[[[X,Y] || X <- [R], Y <- "123456789"] || R <- "ABCDEFGHI"] > ? ?). > box_squares() -> > ? ?%% All the square names for each box. > ? ?ct_expand:term( > ? ? ? ?[[[X,Y] || X <- R, Y <- C] || R <- ["ABC", "DEF", "GHI"], C <- ["123", "456", "789"]] > ? ?). > > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED] > Sent: Saturday, March 26, 2011 4:39 PM > To: Evans, Matthew > Cc: Andreas Pauley; erlang-questions@REDACTED > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > BTW, beside looking into reducing eleminations (which requires deep dive), if u do some time profiling using eprof u will find that most of the time spent actually in the cross functions > sudoku:'-cross/2-lc$^1/1-1-'/4 ? ? ? ? ? ? ? ? ? ? ? 96558660 ?37.46 ?35024918 ?[ ? ? ?0.36] > > On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED<mailto:spawn.think@REDACTED>> wrote: > actually instead of doing > > > > ? ?NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > > > > ? ?PeerSet = sets:from_list(NonUniquePeers), > > > > ? ?PeersWithSelf = sets:to_list(PeerSet), > > > > > > > > > can't u just do > > > PeersWithSelf = lists:usort(NonUniquePeers). > > > > ? > > > > > > > > > On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > Good call....gb_sets should be faster. Compiled to native it runs faster still > > 9> sudoku:print_results("top95.txt", "\n"). > Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min 1792). > ok > > ________________________________________ > From: Ahmed Omar [spawn.think@REDACTED<mailto:spawn.think@REDACTED>] > Sent: Saturday, March 26, 2011 3:08 PM > To: Andreas Pauley > Cc: Evans, Matthew; erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang compared to Python > > using gb_sets instead of sets could decrease eliminations a bit and give u some boost. However, i didn't dive deeper into the code or the algorithm. > > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED<mailto:apauley@REDACTED><mailto:apauley@REDACTED<mailto:apauley@REDACTED>>> wrote: > Thanks, I've changed the compile options and this definitely makes it > somewhat faster: > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > > Strangely the native flag is not documented here: > http://www.erlang.org/doc/man/compile.html#file-2 > > The more interesting improvement would be to decrease the number of > eliminations performed, but for that I'll have to go deep into the > code :-) > > > On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED><mailto:mevans@REDACTED<mailto:mevans@REDACTED>>> wrote: >> Without going deep into the code, one thing to try is compile with the native flag (running the tests from the VM shell here): >> >> Without native set: >> 18> c(sudoku). >> {ok,sudoku} >> 19> ?sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> ok >> >> >> With native set: >> 20> c(sudoku,[native]). >> {ok,sudoku} >> 21> ?sudoku:print_results("top95.txt", "\n"). >> Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> (922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> >> >> >> -----Original Message----- >> From: erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>> [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED><mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>>] On Behalf Of Andreas Pauley >> Sent: Friday, March 25, 2011 8:15 AM >> To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> Subject: [erlang-questions 4] A sudoku solver in Erlang compared to Python >> >> Hi all, >> >> In my quest to learn Erlang I've translated Peter Norvig's sudoku >> solver into Erlang: >> https://github.com/apauley/sudoku-in-erlang >> >> For comparison, my slightly modified version of Norvig's Python code >> can be found here: >> https://github.com/apauley/sudoku-by-norvig >> >> I like the pattern matching, it was fun to do the entire solution >> without a single if or case statement :-) >> Something that bothers me though is that the Python solution seems to >> be faster, even after I've made the Erlang solution use multiple >> processors. >> >> In order to compare the two solutions I counted the number of >> eliminations each performed. >> The eliminate function is the core of the solution, for example >> assigning a single value to a square is implemented as the elimination >> of all other values. >> >> With the Erlang implementation I get: >> sudoku-in-erlang$ ./sudoku >> All tests passed :-) >> Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min 1770). >> Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, min 1797). >> Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >> ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min 1781). >> >> And with the Python implementation: >> sudoku-by-norvig$ ./sudoku.py >> All tests pass. >> Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> ?(33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). >> Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >> ?(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> >> So according to the stats above, the Python solution performs less >> computations when given exactly the same input. >> The Erlang code is as close to the Python as I could make it, I've >> done more or less a direct translation of the algorithms used. >> >> I suspect that there are some lazy evaluation happening in the Python >> version, possibly generators, although I haven't pinpointed it yet. >> >> How can I improve my Erlang code in this solution? >> >> Kind regards, >> Andreas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED><mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think<http://twitter.com/#!/spawn_think> > > From nick.sfx.1@REDACTED Mon Mar 28 13:42:38 2011 From: nick.sfx.1@REDACTED (Nick S) Date: Mon, 28 Mar 2011 14:42:38 +0300 Subject: [erlang-questions 80] Re: [erlang-questions] otp/supervisor immediately goes to "shutdown" In-Reply-To: <AANLkTimc=uJ2YqPL+2p93hGbB0Kau3XpyYPg-d7bGiY5@mail.gmail.com> References: <AANLkTimPF8GvJWMf2sy7UUb2sLi397-=1bHEgiShRFva@mail.gmail.com> <AANLkTimc=uJ2YqPL+2p93hGbB0Kau3XpyYPg-d7bGiY5@mail.gmail.com> Message-ID: <AANLkTi=w-wHqtzvwF3YDcM0YXj=9mj7bNN3ZJ0EUCF2T@mail.gmail.com> Thanks Jesper, this all works now! Still one question. Is there a way for supervisor to know PIDs of worker processes, in this case ----------------------------- module(my_sup). init() -> {ok,{{one_for_one, ......}, [WorkerSpecs]}}. ----------------------------- One way for supervisor would be to pass self() in argument, and worker process would send their self() back, but this would be some code in Supervisor. On Tue, Mar 8, 2011 at 1:23 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Mar 8, 2011 at 11:50, Nick S <nick.sfx.1@REDACTED> wrote: > > Hi, > > > > Just hoping to get some help on my query > > > > http://forum.trapexit.org/viewtopic.php?t=19641 > > Your problem is probably that your start_link/1 code in the db module > is written as: > > start_link([S]) -> > > where you should have written > > start_link(S) -> > > as the list is "unpacked" in the supervisor (probably due to an > apply/3 form). The way you can see this is to up your debugging > information on the system. Eventually boot with SASL enabled, and also > look at the spawn options, notably the debug tracing (see gen_server > and sys module descriptions for enabling this). My guess is though > that in this case, SASL is enough as you will probably be hitting a > function_clause error when the supervisor tries to call (the > equivalent of apply(db, start_link, [S]). > > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/538f2619/attachment.htm> From jesper.louis.andersen@REDACTED Mon Mar 28 13:59:44 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 28 Mar 2011 13:59:44 +0200 Subject: [erlang-questions 81] Re: [erlang-questions] otp/supervisor immediately goes to "shutdown" In-Reply-To: <AANLkTi=w-wHqtzvwF3YDcM0YXj=9mj7bNN3ZJ0EUCF2T@mail.gmail.com> References: <AANLkTimPF8GvJWMf2sy7UUb2sLi397-=1bHEgiShRFva@mail.gmail.com> <AANLkTimc=uJ2YqPL+2p93hGbB0Kau3XpyYPg-d7bGiY5@mail.gmail.com> <AANLkTi=w-wHqtzvwF3YDcM0YXj=9mj7bNN3ZJ0EUCF2T@mail.gmail.com> Message-ID: <AANLkTimPGVio_t=mh4nsXN0GAZbuHoK61saf2cXLt5e8@mail.gmail.com> On Mon, Mar 28, 2011 at 13:42, Nick S <nick.sfx.1@REDACTED> wrote: > Thanks Jesper, this all works now! > Still one question. Is there a way for supervisor to know PIDs of worker > processes, in this case There are a couple of options: If the process is singular in the system, you can register/2 it. If the process is not singular, you can use a process registry to store its pid() and have other processes look up the Pid in the registry. gproc is one such registry, but you can also write one yourself with ETS tables, or use the global register facility which can map arbitrary terms to pid()'s (local registry can only map atoms). Finally, you can ask the supervisor: supervisor:which_children/1 -- J. From hm@REDACTED Mon Mar 28 14:29:47 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 28 Mar 2011 14:29:47 +0200 Subject: [erlang-questions 82] Re: code:lib_dir and .ez archives In-Reply-To: <AANLkTimhcFEq4+A25+1iRFXvR9dendnDGasbWBBLnzFL@mail.gmail.com> References: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> <1301248066671-3409584.post@n4.nabble.com> <AANLkTimhcFEq4+A25+1iRFXvR9dendnDGasbWBBLnzFL@mail.gmail.com> Message-ID: <AANLkTimG67+qub+6_ertuAQAxN+P-UP3qK5gDwNZ5Ve_@mail.gmail.com> On Mon, Mar 28, 2011 at 8:52 AM, Max Lapshin <max.lapshin@REDACTED> wrote: > There is other problem with escript: it cannot go background. > It would be very convenient for me to distribute erlyvideo in small > single-file launch scripts, but I can't do it because it doesn't allow > to go background. Yes, you can (forcefully) put it in background with the -detached option: %%! -detached It is ugly, but it works. Per Hedeland has however implemented a much better solution called delayed detach, where the detached Erlang node keeps a socket to its parent process so it may print a decent error message and set an exit code if the startup fails. It is up to the application to decide whether the detach should completed or if it is an error case. The code has been contributed to the OTP group a long time ago, but was at that timepoint not included in the distribution due to lack of time. The code has proven to work very well. Delayed detach would be a perfect fit for usage with escripts. /H?kan From apauley@REDACTED Mon Mar 28 15:12:02 2011 From: apauley@REDACTED (Andreas Pauley) Date: Mon, 28 Mar 2011 15:12:02 +0200 Subject: [erlang-questions 83] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTinipQgWi8F0LYFtz==n=f7-snG5BVArcC2+svz1@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinipQgWi8F0LYFtz==n=f7-snG5BVArcC2+svz1@mail.gmail.com> Message-ID: <AANLkTi=kZ1CjmdDk0fbyG+2Gw8Uqn=wXPXhFcAW36Nje@mail.gmail.com> Cool, good catch! https://github.com/apauley/sudoku-in-erlang/commit/9125147621faeb11cd46cea5100b2e1e88ff9732 This reduces the elimination count significantly :-) Solved 50 of 50 puzzles from easy50.txt in 1.286811 secs (38.86 Hz) (32833 total eliminations, avg 656.66, median 648, max 808, min 648). Solved 95 of 95 puzzles from top95.txt in 8.572484 secs (11.08 Hz) (210794 total eliminations, avg 2218.88, median 1536, max 10664, min 648). Solved 11 of 11 puzzles from hardest.txt in 0.391032 secs (28.13 Hz) (9516 total eliminations, avg 865.09, median 813, max 1227, min 648). On Sun, Mar 27, 2011 at 11:23 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > running test after fixing the counter gives the following > Solved 95 of 95 puzzles from top95.txt in 2.235227 secs (42.50 Hz) > (210794 total eliminations, avg 2218.88, median 1536, max 10664, min 648). > > On Sun, Mar 27, 2011 at 11:21 PM, Ahmed Omar <spawn.think@REDACTED> wrote: >> >> No problem, as said before u can also replace these lines : >> PeerSet = gb_sets:from_list(NonUniquePeers), >> PeersWithSelf = gb_sets:to_list(PeerSet), >> lists:delete(Square, PeersWithSelf). >> with just >> lists:delete(Square,?lists:usort(NonUniquePeers)). >> (don't forget to check the rest of the thread too ;)) >> On Sun, Mar 27, 2011 at 11:02 PM, Andreas Pauley <apauley@REDACTED> >> wrote: >>> >>> Thanks! >>> I didn't think about the sets at all :-) >>> >>> >>> https://github.com/apauley/sudoku-in-erlang/commit/a02fbcd001a9fbd876725008d58c52fcff9872d9 >>> >>> On Sat, Mar 26, 2011 at 9:08 PM, Ahmed Omar <spawn.think@REDACTED> >>> wrote: >>> > using gb_sets instead of sets could decrease eliminations a bit and >>> > give u >>> > some boost. However, i didn't dive deeper into the code or the >>> > algorithm. >>> > >>> > On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley <apauley@REDACTED> >>> > wrote: >>> >> >>> >> Thanks, I've changed the compile options and this definitely makes it >>> >> somewhat faster: >>> >> >>> >> >>> >> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >>> >> >>> >> Strangely the native flag is not documented here: >>> >> http://www.erlang.org/doc/man/compile.html#file-2 >>> >> >>> >> The more interesting improvement would be to decrease the number of >>> >> eliminations performed, but for that I'll have to go deep into the >>> >> code :-) >>> >> >>> >> >>> >> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew <mevans@REDACTED> >>> >> wrote: >>> >> > Without going deep into the code, one thing to try is compile with >>> >> > the >>> >> > native flag (running the tests from the VM shell here): >>> >> > >>> >> > Without native set: >>> >> > 18> c(sudoku). >>> >> > {ok,sudoku} >>> >> > 19> ?sudoku:print_results("top95.txt", "\n"). >>> >> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >>> >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>> >> > 1797). >>> >> > ok >>> >> > >>> >> > >>> >> > With native set: >>> >> > 20> c(sudoku,[native]). >>> >> > {ok,sudoku} >>> >> > 21> ?sudoku:print_results("top95.txt", "\n"). >>> >> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >>> >> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>> >> > 1797). >>> >> > >>> >> > >>> >> > >>> >> > -----Original Message----- >>> >> > From: erlang-questions-bounces@REDACTED >>> >> > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Andreas >>> >> > Pauley >>> >> > Sent: Friday, March 25, 2011 8:15 AM >>> >> > To: erlang-questions@REDACTED >>> >> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >>> >> > Python >>> >> > >>> >> > Hi all, >>> >> > >>> >> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >>> >> > solver into Erlang: >>> >> > https://github.com/apauley/sudoku-in-erlang >>> >> > >>> >> > For comparison, my slightly modified version of Norvig's Python code >>> >> > can be found here: >>> >> > https://github.com/apauley/sudoku-by-norvig >>> >> > >>> >> > I like the pattern matching, it was fun to do the entire solution >>> >> > without a single if or case statement :-) >>> >> > Something that bothers me though is that the Python solution seems >>> >> > to >>> >> > be faster, even after I've made the Erlang solution use multiple >>> >> > processors. >>> >> > >>> >> > In order to compare the two solutions I counted the number of >>> >> > eliminations each performed. >>> >> > The eliminate function is the core of the solution, for example >>> >> > assigning a single value to a square is implemented as the >>> >> > elimination >>> >> > of all other values. >>> >> > >>> >> > With the Erlang implementation I get: >>> >> > sudoku-in-erlang$ ./sudoku >>> >> > All tests passed :-) >>> >> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >>> >> > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min >>> >> > 1770). >>> >> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >>> >> > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, >>> >> > min >>> >> > 1797). >>> >> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >>> >> > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min >>> >> > 1781). >>> >> > >>> >> > And with the Python implementation: >>> >> > sudoku-by-norvig$ ./sudoku.py >>> >> > All tests pass. >>> >> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >>> >> > ?(33059 total eliminations, avg 661.00, median 648, max 830, min >>> >> > 648). >>> >> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >>> >> > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, >>> >> > min >>> >> > 648). >>> >> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >>> >> > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min >>> >> > 648). >>> >> > >>> >> > So according to the stats above, the Python solution performs less >>> >> > computations when given exactly the same input. >>> >> > The Erlang code is as close to the Python as I could make it, I've >>> >> > done more or less a direct translation of the algorithms used. >>> >> > >>> >> > I suspect that there are some lazy evaluation happening in the >>> >> > Python >>> >> > version, possibly generators, although I haven't pinpointed it yet. >>> >> > >>> >> > How can I improve my Erlang code in this solution? >>> >> > >>> >> > Kind regards, >>> >> > Andreas >>> >> > _______________________________________________ >>> >> > erlang-questions mailing list >>> >> > erlang-questions@REDACTED >>> >> > http://erlang.org/mailman/listinfo/erlang-questions >>> >> > >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >>> > >>> > -- >>> > Best Regards, >>> > - Ahmed Omar >>> > http://nl.linkedin.com/in/adiaa >>> > Follow me on twitter >>> > @spawn_think >>> > >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From apauley@REDACTED Mon Mar 28 15:22:17 2011 From: apauley@REDACTED (Andreas Pauley) Date: Mon, 28 Mar 2011 15:22:17 +0200 Subject: [erlang-questions 84] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> Message-ID: <AANLkTikQr8-OLPWUUn9Fi=xP9OTJhyeqYUABXhC=Hr=x@mail.gmail.com> On Sun, Mar 27, 2011 at 3:28 PM, Kresten Krab Thorup <krab@REDACTED> wrote: > For reference, I tried to run this with Erjang and R14B01 (with BEAN and HiPE/o3) on my machine ... > > Oh, and I changed the square names to be atoms in stead of [X,Y] pairs, which speeds up things quite a lot too ... e.g. : I'd love to use atoms as square names, just because it looks cleaner. However, I'm getting a severe performance hit when I use list_to_atom/1 on each square name. Can you show me how you did it in conjunction with ct_expand? Thanks! Andreas From cbenac@REDACTED Mon Mar 28 15:28:13 2011 From: cbenac@REDACTED (Clara Benac Earle) Date: Mon, 28 Mar 2011 15:28:13 +0200 Subject: [erlang-questions 85] [ANN] Reminder: Madrid Spring Erlounge Message-ID: <4D908CED.3000303@fi.upm.es> This is a reminder for the Madrid Spring Erlounge. Date Wednesday 30th of March 2011 Time 18:30 - 20:30 (talk starts at 19:00) Place Sala de Grados (1st floor of the Facultad de Inform?tica of the Universidad Complutense of Madrid) http://gpd.sip.ucm.es/fraguas/wflp06/campus_ucm.jpg Speaker: Juan Puig, Systems Engineer at Erlang Solutions Ltd. (http://www.erlang-solutions.com/) To try and get more Spanish people interested in Erlang, the talk will be in Spanish. *Resumen de la conferencia* Desde que Erlang vi? la luz en 1998 como proyecto de software libre, no ha dejado de crecer. Como muestra de ello, haremos un repaso a la actualidad del lenguaje como tecnolog?a, as? como a su comunidad de open source, y de qu? manera Erlang Solutions contribuye a ella con sus proyectos p?blicos m?s importantes. See you there! Clara -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/dae27569/attachment.htm> From apauley@REDACTED Mon Mar 28 15:32:20 2011 From: apauley@REDACTED (Andreas Pauley) Date: Mon, 28 Mar 2011 15:32:20 +0200 Subject: [erlang-questions 86] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> Message-ID: <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> Hi Tristan, I tried the code in your repo, but it wasn't significantly faster on my machine. It was interesting though to go through your refactorings. I see you changed the signature for solve so that the empty puzzle and the list of square names are passed in from the start. Maybe I should try this approach in conjunction with the conversion to atoms... Regards, Andreas On Sat, Mar 26, 2011 at 10:56 PM, Tristan Sloughter <tristan.sloughter@REDACTED> wrote: > With a few refactorings (https://github.com/tsloughter/sudoku-in-erlang -- > note I used ewl_plists just to make the code simpler: > https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile with > native, hipe o3 it beats python's: > $ ./sudoku.py > All tests pass. > Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) > ??(33059 total eliminations, avg 661.00, median 648, max 830, min 648). > Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) > ??(221997 total eliminations, avg 2336.00, median 1492, max 11512, min 648). > Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) > ??(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > $ ./sudoku > All tests passed :-) > Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) > ??(92538 total eliminations, avg 1850.76, median 1811, max 2628, min 1767). > Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) > ??(901201 total eliminations, avg 9486.33, median 6267, max 56820, min > 1792). > Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) > ??(33653 total eliminations, avg 3059.36, median 3023, max 5346, min 1786). > I ran a number of times, those are pretty much what it always comes out to. > But I haven't really tried a full refactoring, I think a number of things > can be reduced to less iterations. > > Tristan > On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> wrote: >> >> BTW, beside looking into reducing eleminations (which requires deep dive), >> if u do some time profiling using eprof u will find that most of the time >> spent?actually?in the cross functions >> sudoku:'-cross/2-lc$^1/1-1-'/4 ? ? ? ? ? ? ? ? ? ? ? 96558660 ?37.46 >> ?35024918 ?[ ? ? ?0.36] >> >> On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> wrote: >>> >>> actually instead of doing >>> >>> ????NonUniquePeers = shallow_flatten([S || S <- units(Square)]), >>> >>> >>> ????PeerSet = sets:from_list(NonUniquePeers), >>> >>> >>> ????PeersWithSelf = sets:to_list(PeerSet), >>> >>> >>> >>> >>> >>> >>> >>> can't u just do >>> PeersWithSelf = lists:usort(NonUniquePeers). >>> >>> >>> ? >>> >>> >>> >>> >>> >>> >>> >>> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED> >>> wrote: >>>> >>>> Good call....gb_sets should be faster. Compiled to native it runs faster >>>> still >>>> >>>> 9> sudoku:print_results("top95.txt", "\n"). >>>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >>>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >>>> 1792). >>>> ok >>>> >>>> ________________________________________ >>>> From: Ahmed Omar [spawn.think@REDACTED] >>>> Sent: Saturday, March 26, 2011 3:08 PM >>>> To: Andreas Pauley >>>> Cc: Evans, Matthew; erlang-questions@REDACTED >>>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang >>>> compared to Python >>>> >>>> using gb_sets instead of sets could decrease eliminations a bit and give >>>> u some boost. However, i didn't dive deeper into the code or the algorithm. >>>> >>>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley >>>> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: >>>> Thanks, I've changed the compile options and this definitely makes it >>>> somewhat faster: >>>> >>>> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >>>> >>>> Strangely the native flag is not documented here: >>>> http://www.erlang.org/doc/man/compile.html#file-2 >>>> >>>> The more interesting improvement would be to decrease the number of >>>> eliminations performed, but for that I'll have to go deep into the >>>> code :-) >>>> >>>> >>>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew >>>> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: >>>> > Without going deep into the code, one thing to try is compile with the >>>> > native flag (running the tests from the VM shell here): >>>> > >>>> > Without native set: >>>> > 18> c(sudoku). >>>> > {ok,sudoku} >>>> > 19> ?sudoku:print_results("top95.txt", "\n"). >>>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>>> > 1797). >>>> > ok >>>> > >>>> > >>>> > With native set: >>>> > 20> c(sudoku,[native]). >>>> > {ok,sudoku} >>>> > 21> ?sudoku:print_results("top95.txt", "\n"). >>>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>>> > 1797). >>>> > >>>> > >>>> > >>>> > -----Original Message----- >>>> > From: >>>> > erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED> >>>> > [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>] >>>> > On Behalf Of Andreas Pauley >>>> > Sent: Friday, March 25, 2011 8:15 AM >>>> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >>>> > Python >>>> > >>>> > Hi all, >>>> > >>>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >>>> > solver into Erlang: >>>> > https://github.com/apauley/sudoku-in-erlang >>>> > >>>> > For comparison, my slightly modified version of Norvig's Python code >>>> > can be found here: >>>> > https://github.com/apauley/sudoku-by-norvig >>>> > >>>> > I like the pattern matching, it was fun to do the entire solution >>>> > without a single if or case statement :-) >>>> > Something that bothers me though is that the Python solution seems to >>>> > be faster, even after I've made the Erlang solution use multiple >>>> > processors. >>>> > >>>> > In order to compare the two solutions I counted the number of >>>> > eliminations each performed. >>>> > The eliminate function is the core of the solution, for example >>>> > assigning a single value to a square is implemented as the elimination >>>> > of all other values. >>>> > >>>> > With the Erlang implementation I get: >>>> > sudoku-in-erlang$ ./sudoku >>>> > All tests passed :-) >>>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >>>> > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min >>>> > 1770). >>>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >>>> > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, min >>>> > 1797). >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) >>>> > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min >>>> > 1781). >>>> > >>>> > And with the Python implementation: >>>> > sudoku-by-norvig$ ./sudoku.py >>>> > All tests pass. >>>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >>>> > ?(33059 total eliminations, avg 661.00, median 648, max 830, min 648). >>>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >>>> > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, min >>>> > 648). >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) >>>> > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >>>> > >>>> > So according to the stats above, the Python solution performs less >>>> > computations when given exactly the same input. >>>> > The Erlang code is as close to the Python as I could make it, I've >>>> > done more or less a direct translation of the algorithms used. >>>> > >>>> > I suspect that there are some lazy evaluation happening in the Python >>>> > version, possibly generators, although I haven't pinpointed it yet. >>>> > >>>> > How can I improve my Erlang code in this solution? >>>> > >>>> > Kind regards, >>>> > Andreas >>>> > _______________________________________________ >>>> > erlang-questions mailing list >>>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>>> > http://erlang.org/mailman/listinfo/erlang-questions >>>> > >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> - Ahmed Omar >>>> http://nl.linkedin.com/in/adiaa >>>> Follow me on twitter >>>> @spawn_think<http://twitter.com/#!/spawn_think> >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> - Ahmed Omar >>> http://nl.linkedin.com/in/adiaa >>> Follow me on twitter >>> @spawn_think >> >> >> >> -- >> Best Regards, >> - Ahmed Omar >> http://nl.linkedin.com/in/adiaa >> Follow me on twitter >> @spawn_think >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > From tristan.sloughter@REDACTED Mon Mar 28 15:54:46 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 28 Mar 2011 08:54:46 -0500 Subject: [erlang-questions 87] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> Message-ID: <AANLkTimJJeit4t-x_BsDamDdh49=Babt1NuNCBXAia5u@mail.gmail.com> Yeah, that change didn't seem to improve it but it was something I saw as being run over and over when it could just be a copy. Really the only speed up mine gave was from the switch to gb_sets. And I think the plists may be a slightly faster for some reason... Tristan On Mon, Mar 28, 2011 at 8:32 AM, Andreas Pauley <apauley@REDACTED> wrote: > Hi Tristan, > > I tried the code in your repo, but it wasn't significantly faster on my > machine. > > It was interesting though to go through your refactorings. > I see you changed the signature for solve so that the empty puzzle and > the list of square names are passed in from the start. > Maybe I should try this approach in conjunction with the conversion to > atoms... > > Regards, > Andreas > > On Sat, Mar 26, 2011 at 10:56 PM, Tristan Sloughter > <tristan.sloughter@REDACTED> wrote: > > With a few refactorings (https://github.com/tsloughter/sudoku-in-erlang-- > > note I used ewl_plists just to make the code simpler: > > https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile > with > > native, hipe o3 it beats python's: > > $ ./sudoku.py > > All tests pass. > > Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) > > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > > Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) > > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > > Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) > > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > $ ./sudoku > > All tests passed :-) > > Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) > > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min > 1767). > > Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) > > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > > 1792). > > Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) > > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min > 1786). > > I ran a number of times, those are pretty much what it always comes out > to. > > But I haven't really tried a full refactoring, I think a number of things > > can be reduced to less iterations. > > > > Tristan > > On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> > wrote: > >> > >> BTW, beside looking into reducing eleminations (which requires deep > dive), > >> if u do some time profiling using eprof u will find that most of the > time > >> spent actually in the cross functions > >> sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 > >> 35024918 [ 0.36] > >> > >> On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> > wrote: > >>> > >>> actually instead of doing > >>> > >>> NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > >>> > >>> > >>> PeerSet = sets:from_list(NonUniquePeers), > >>> > >>> > >>> PeersWithSelf = sets:to_list(PeerSet), > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> can't u just do > >>> PeersWithSelf = lists:usort(NonUniquePeers). > >>> > >>> > >>> ? > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED> > >>> wrote: > >>>> > >>>> Good call....gb_sets should be faster. Compiled to native it runs > faster > >>>> still > >>>> > >>>> 9> sudoku:print_results("top95.txt", "\n"). > >>>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > >>>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > >>>> 1792). > >>>> ok > >>>> > >>>> ________________________________________ > >>>> From: Ahmed Omar [spawn.think@REDACTED] > >>>> Sent: Saturday, March 26, 2011 3:08 PM > >>>> To: Andreas Pauley > >>>> Cc: Evans, Matthew; erlang-questions@REDACTED > >>>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang > >>>> compared to Python > >>>> > >>>> using gb_sets instead of sets could decrease eliminations a bit and > give > >>>> u some boost. However, i didn't dive deeper into the code or the > algorithm. > >>>> > >>>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley > >>>> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: > >>>> Thanks, I've changed the compile options and this definitely makes it > >>>> somewhat faster: > >>>> > >>>> > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > >>>> > >>>> Strangely the native flag is not documented here: > >>>> http://www.erlang.org/doc/man/compile.html#file-2 > >>>> > >>>> The more interesting improvement would be to decrease the number of > >>>> eliminations performed, but for that I'll have to go deep into the > >>>> code :-) > >>>> > >>>> > >>>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew > >>>> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > >>>> > Without going deep into the code, one thing to try is compile with > the > >>>> > native flag (running the tests from the VM shell here): > >>>> > > >>>> > Without native set: > >>>> > 18> c(sudoku). > >>>> > {ok,sudoku} > >>>> > 19> sudoku:print_results("top95.txt", "\n"). > >>>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >>>> > 1797). > >>>> > ok > >>>> > > >>>> > > >>>> > With native set: > >>>> > 20> c(sudoku,[native]). > >>>> > {ok,sudoku} > >>>> > 21> sudoku:print_results("top95.txt", "\n"). > >>>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >>>> > 1797). > >>>> > > >>>> > > >>>> > > >>>> > -----Original Message----- > >>>> > From: > >>>> > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED> > >>>> > [mailto:erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>] > >>>> > On Behalf Of Andreas Pauley > >>>> > Sent: Friday, March 25, 2011 8:15 AM > >>>> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > >>>> > Python > >>>> > > >>>> > Hi all, > >>>> > > >>>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku > >>>> > solver into Erlang: > >>>> > https://github.com/apauley/sudoku-in-erlang > >>>> > > >>>> > For comparison, my slightly modified version of Norvig's Python code > >>>> > can be found here: > >>>> > https://github.com/apauley/sudoku-by-norvig > >>>> > > >>>> > I like the pattern matching, it was fun to do the entire solution > >>>> > without a single if or case statement :-) > >>>> > Something that bothers me though is that the Python solution seems > to > >>>> > be faster, even after I've made the Erlang solution use multiple > >>>> > processors. > >>>> > > >>>> > In order to compare the two solutions I counted the number of > >>>> > eliminations each performed. > >>>> > The eliminate function is the core of the solution, for example > >>>> > assigning a single value to a square is implemented as the > elimination > >>>> > of all other values. > >>>> > > >>>> > With the Erlang implementation I get: > >>>> > sudoku-in-erlang$ ./sudoku > >>>> > All tests passed :-) > >>>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > >>>> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > >>>> > 1770). > >>>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, > min > >>>> > 1797). > >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > >>>> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > >>>> > 1781). > >>>> > > >>>> > And with the Python implementation: > >>>> > sudoku-by-norvig$ ./sudoku.py > >>>> > All tests pass. > >>>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > >>>> > (33059 total eliminations, avg 661.00, median 648, max 830, min > 648). > >>>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > >>>> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, > min > >>>> > 648). > >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > >>>> > (9436 total eliminations, avg 857.00, median 817, max 1198, min > 648). > >>>> > > >>>> > So according to the stats above, the Python solution performs less > >>>> > computations when given exactly the same input. > >>>> > The Erlang code is as close to the Python as I could make it, I've > >>>> > done more or less a direct translation of the algorithms used. > >>>> > > >>>> > I suspect that there are some lazy evaluation happening in the > Python > >>>> > version, possibly generators, although I haven't pinpointed it yet. > >>>> > > >>>> > How can I improve my Erlang code in this solution? > >>>> > > >>>> > Kind regards, > >>>> > Andreas > >>>> > _______________________________________________ > >>>> > erlang-questions mailing list > >>>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> > http://erlang.org/mailman/listinfo/erlang-questions > >>>> > > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>>> > >>>> > >>>> -- > >>>> Best Regards, > >>>> - Ahmed Omar > >>>> http://nl.linkedin.com/in/adiaa > >>>> Follow me on twitter > >>>> @spawn_think<http://twitter.com/#!/spawn_think> > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, > >>> - Ahmed Omar > >>> http://nl.linkedin.com/in/adiaa > >>> Follow me on twitter > >>> @spawn_think > >> > >> > >> > >> -- > >> Best Regards, > >> - Ahmed Omar > >> http://nl.linkedin.com/in/adiaa > >> Follow me on twitter > >> @spawn_think > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/f4800741/attachment.htm> From max.lapshin@REDACTED Mon Mar 28 16:01:44 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 28 Mar 2011 18:01:44 +0400 Subject: [erlang-questions 88] Re: code:lib_dir and .ez archives In-Reply-To: <AANLkTimG67+qub+6_ertuAQAxN+P-UP3qK5gDwNZ5Ve_@mail.gmail.com> References: <14B8CD70-51B9-4420-892F-BB85D2FA5E41@erlang-solutions.com> <1301248066671-3409584.post@n4.nabble.com> <AANLkTimhcFEq4+A25+1iRFXvR9dendnDGasbWBBLnzFL@mail.gmail.com> <AANLkTimG67+qub+6_ertuAQAxN+P-UP3qK5gDwNZ5Ve_@mail.gmail.com> Message-ID: <AANLkTinhrrtFTcEnvk8AKP9NoKSoj_e6GADL_VEjioiG@mail.gmail.com> 2011/3/28 H?kan Mattsson <hm@REDACTED>: > On Mon, Mar 28, 2011 at 8:52 AM, Max Lapshin <max.lapshin@REDACTED> wrote: >> There is other problem with escript: it cannot go background. >> It would be very convenient for me to distribute erlyvideo in small >> single-file launch scripts, but I can't do it because it doesn't allow >> to go background. > > Yes, you can (forcefully) put it in background with the -detached option: > > ? ?%%! -detached > > It is ugly, but it works. > I'll check it. In fact, I need os:daemonize() =) From spawn.think@REDACTED Mon Mar 28 16:39:36 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 28 Mar 2011 16:39:36 +0200 Subject: [erlang-questions 89] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> Message-ID: <AANLkTimLHaUJr0d5qmX53-jvBNKdV+rZH9SiJCwJEd3O@mail.gmail.com> Hi Andreas, For some reason, lists:usort is your friend again! Looking into the code deeper, one line can give u huge boost! You compare the unit values with digits with (length(UnitValues) == 9) and (gb_sets:from_list(UnitValues) == gb_sets:from_list(digits())). I understand (correct me if i'm wrong) u check first the length to make sure it's 9 digits then u convert to sets to make sure they r unique and compare them. But if you think about it, if the unique list of values is of length 9, u don't need more comparisons! so u can actually just do (length(lists:usort(UnitValues)==9) before the change: sudoku:print_results("top95.txt"). Solved 95 of 95 puzzles from top95.txt in 1.937175 secs (49.04 Hz) (210794 total eliminations, avg 2218.88, median 1536, max 10664, min 648). after the change : sudoku:print_results("top95.txt"). Solved 95 of 95 puzzles from top95.txt in 0.382337 secs (248.47 Hz) (45002 total eliminations, avg 473.71, median 479, max 518, min 427). On Mon, Mar 28, 2011 at 3:32 PM, Andreas Pauley <apauley@REDACTED> wrote: > Hi Tristan, > > I tried the code in your repo, but it wasn't significantly faster on my > machine. > > It was interesting though to go through your refactorings. > I see you changed the signature for solve so that the empty puzzle and > the list of square names are passed in from the start. > Maybe I should try this approach in conjunction with the conversion to > atoms... > > Regards, > Andreas > > On Sat, Mar 26, 2011 at 10:56 PM, Tristan Sloughter > <tristan.sloughter@REDACTED> wrote: > > With a few refactorings (https://github.com/tsloughter/sudoku-in-erlang-- > > note I used ewl_plists just to make the code simpler: > > https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile > with > > native, hipe o3 it beats python's: > > $ ./sudoku.py > > All tests pass. > > Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) > > (33059 total eliminations, avg 661.00, median 648, max 830, min 648). > > Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) > > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > 648). > > Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) > > (9436 total eliminations, avg 857.00, median 817, max 1198, min 648). > > $ ./sudoku > > All tests passed :-) > > Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) > > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min > 1767). > > Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) > > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > > 1792). > > Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) > > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min > 1786). > > I ran a number of times, those are pretty much what it always comes out > to. > > But I haven't really tried a full refactoring, I think a number of things > > can be reduced to less iterations. > > > > Tristan > > On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> > wrote: > >> > >> BTW, beside looking into reducing eleminations (which requires deep > dive), > >> if u do some time profiling using eprof u will find that most of the > time > >> spent actually in the cross functions > >> sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 > >> 35024918 [ 0.36] > >> > >> On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> > wrote: > >>> > >>> actually instead of doing > >>> > >>> NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > >>> > >>> > >>> PeerSet = sets:from_list(NonUniquePeers), > >>> > >>> > >>> PeersWithSelf = sets:to_list(PeerSet), > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> can't u just do > >>> PeersWithSelf = lists:usort(NonUniquePeers). > >>> > >>> > >>> ? > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED> > >>> wrote: > >>>> > >>>> Good call....gb_sets should be faster. Compiled to native it runs > faster > >>>> still > >>>> > >>>> 9> sudoku:print_results("top95.txt", "\n"). > >>>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > >>>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > >>>> 1792). > >>>> ok > >>>> > >>>> ________________________________________ > >>>> From: Ahmed Omar [spawn.think@REDACTED] > >>>> Sent: Saturday, March 26, 2011 3:08 PM > >>>> To: Andreas Pauley > >>>> Cc: Evans, Matthew; erlang-questions@REDACTED > >>>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang > >>>> compared to Python > >>>> > >>>> using gb_sets instead of sets could decrease eliminations a bit and > give > >>>> u some boost. However, i didn't dive deeper into the code or the > algorithm. > >>>> > >>>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley > >>>> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: > >>>> Thanks, I've changed the compile options and this definitely makes it > >>>> somewhat faster: > >>>> > >>>> > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > >>>> > >>>> Strangely the native flag is not documented here: > >>>> http://www.erlang.org/doc/man/compile.html#file-2 > >>>> > >>>> The more interesting improvement would be to decrease the number of > >>>> eliminations performed, but for that I'll have to go deep into the > >>>> code :-) > >>>> > >>>> > >>>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew > >>>> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > >>>> > Without going deep into the code, one thing to try is compile with > the > >>>> > native flag (running the tests from the VM shell here): > >>>> > > >>>> > Without native set: > >>>> > 18> c(sudoku). > >>>> > {ok,sudoku} > >>>> > 19> sudoku:print_results("top95.txt", "\n"). > >>>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >>>> > 1797). > >>>> > ok > >>>> > > >>>> > > >>>> > With native set: > >>>> > 20> c(sudoku,[native]). > >>>> > {ok,sudoku} > >>>> > 21> sudoku:print_results("top95.txt", "\n"). > >>>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, min > >>>> > 1797). > >>>> > > >>>> > > >>>> > > >>>> > -----Original Message----- > >>>> > From: > >>>> > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED> > >>>> > [mailto:erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>] > >>>> > On Behalf Of Andreas Pauley > >>>> > Sent: Friday, March 25, 2011 8:15 AM > >>>> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to > >>>> > Python > >>>> > > >>>> > Hi all, > >>>> > > >>>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku > >>>> > solver into Erlang: > >>>> > https://github.com/apauley/sudoku-in-erlang > >>>> > > >>>> > For comparison, my slightly modified version of Norvig's Python code > >>>> > can be found here: > >>>> > https://github.com/apauley/sudoku-by-norvig > >>>> > > >>>> > I like the pattern matching, it was fun to do the entire solution > >>>> > without a single if or case statement :-) > >>>> > Something that bothers me though is that the Python solution seems > to > >>>> > be faster, even after I've made the Erlang solution use multiple > >>>> > processors. > >>>> > > >>>> > In order to compare the two solutions I counted the number of > >>>> > eliminations each performed. > >>>> > The eliminate function is the core of the solution, for example > >>>> > assigning a single value to a square is implemented as the > elimination > >>>> > of all other values. > >>>> > > >>>> > With the Erlang implementation I get: > >>>> > sudoku-in-erlang$ ./sudoku > >>>> > All tests passed :-) > >>>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) > >>>> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, min > >>>> > 1770). > >>>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) > >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, > min > >>>> > 1797). > >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 Hz) > >>>> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, min > >>>> > 1781). > >>>> > > >>>> > And with the Python implementation: > >>>> > sudoku-by-norvig$ ./sudoku.py > >>>> > All tests pass. > >>>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) > >>>> > (33059 total eliminations, avg 661.00, median 648, max 830, min > 648). > >>>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) > >>>> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, > min > >>>> > 648). > >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 Hz) > >>>> > (9436 total eliminations, avg 857.00, median 817, max 1198, min > 648). > >>>> > > >>>> > So according to the stats above, the Python solution performs less > >>>> > computations when given exactly the same input. > >>>> > The Erlang code is as close to the Python as I could make it, I've > >>>> > done more or less a direct translation of the algorithms used. > >>>> > > >>>> > I suspect that there are some lazy evaluation happening in the > Python > >>>> > version, possibly generators, although I haven't pinpointed it yet. > >>>> > > >>>> > How can I improve my Erlang code in this solution? > >>>> > > >>>> > Kind regards, > >>>> > Andreas > >>>> > _______________________________________________ > >>>> > erlang-questions mailing list > >>>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> > http://erlang.org/mailman/listinfo/erlang-questions > >>>> > > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>>> > >>>> > >>>> -- > >>>> Best Regards, > >>>> - Ahmed Omar > >>>> http://nl.linkedin.com/in/adiaa > >>>> Follow me on twitter > >>>> @spawn_think<http://twitter.com/#!/spawn_think> > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, > >>> - Ahmed Omar > >>> http://nl.linkedin.com/in/adiaa > >>> Follow me on twitter > >>> @spawn_think > >> > >> > >> > >> -- > >> Best Regards, > >> - Ahmed Omar > >> http://nl.linkedin.com/in/adiaa > >> Follow me on twitter > >> @spawn_think > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/5f5c598d/attachment.htm> From raimo+erlang-questions@REDACTED Mon Mar 28 16:49:22 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 28 Mar 2011 16:49:22 +0200 Subject: [erlang-questions 90] Re: New Mailing list software In-Reply-To: <20110328075701.GA6079@mulga.csse.unimelb.edu.au> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <4D8D0A5E.3000208@telegraphics.com.au> <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> <20110328073551.GB13069@erix.ericsson.se> <20110328075701.GA6079@mulga.csse.unimelb.edu.au> Message-ID: <20110328144922.GA26511@erix.ericsson.se> On Mon, Mar 28, 2011 at 06:57:01PM +1100, Jeff Schultz wrote: > On Mon, Mar 28, 2011 at 09:35:51AM +0200, Raimo Niskanen wrote: > > Yes. It is auto numbering of posts. I thought it would be something to > > facilitate threading in the clients, but seems to cause confusion. > > > I think I will shut it off. > > Yes please! > > > BTW, in attempting to do so for myself, I find that the "email me my > password" button seems to not work. I will take a look at that too... > > > Jeff Schultz -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From per@REDACTED Mon Mar 28 17:06:44 2011 From: per@REDACTED (Per Hedeland) Date: Mon, 28 Mar 2011 17:06:44 +0200 (CEST) Subject: [erlang-questions] Re: [erlang-questions 82] Re: code:lib_dir and .ez archives In-Reply-To: <AANLkTimG67+qub+6_ertuAQAxN+P-UP3qK5gDwNZ5Ve_@mail.gmail.com> Message-ID: <201103281506.p2SF6is9049367@pluto.hedeland.org> H?kan Mattsson wrote: > >Per Hedeland has however implemented a much better solution called delayed >detach, where the detached Erlang node keeps a socket to its parent process so >it may print a decent error message and set an exit code if the >startup fails. Actually I didn't implement it, just hacked it when needed...:-) > The code has been contributed to the OTP group a long time ago, but was >at that timepoint not included in the distribution due to lack of >time. The code has >proven to work very well. I guess we could try to dig it out of our repository again, it has gotten a bit mixed up with Tail-f-specific stuff over the years though. --Per From hm@REDACTED Mon Mar 28 17:10:25 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 28 Mar 2011 17:10:25 +0200 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <4D902590.8060409@m5net.com> References: <4D902590.8060409@m5net.com> Message-ID: <AANLkTikmH3+RFUu8g7tv5GDxYXeZZ6RUpe=HebMLh3W2@mail.gmail.com> On Mon, Mar 28, 2011 at 8:07 AM, Sam Bobroff <sam@REDACTED> wrote: > Hi Erlangers, > > I've been trying to work out how to "properly" build Erlang releases (on > a Linux system, to be deployed on another Linux system), and I'm getting > nowhere fast so I'm hoping someone here will have some useful pointers > or hints to help me out. I'm using Erlang R14B, but I assume the > problems I'm having are to do with understanding, rather than bugs so > the version probably doesn't matter much ;-) > > For context, I've read a lot of the online Erlang documentation > including the sections on systools, reltool, sys, rel (and I've read the > Rebar documentation as well, I'll get to that later) and I'm familiar > with setting up OTP applications but this issue is confusing me more and > more: there doesn't seem to be any kind of overview that tells me how > the various pieces fit together or what I'm supposed to do. Is there > some standard document I'm missing? No, better documentation is needed. > "All" I want to do is go from a set of Erlang source, representing one > or more applications (presumably in a set of src and ebin dirs), to an > installed Erlang system that will start those applications. (Which I'm > obviously intending to package up for distribution but that's a separate > issue.) > > I realize I'm rambling so I'll try to phrase some specific questions: > > Is reltool:create_target() the correct way to build a release? Yes. > If so, how do I get reltool:create_target() to include my apps? I've > tried using lib_dirs and -pa in various combinations but I always get > "Release NAME uses non existing application APP". I haven't yet tried > installing my application directly to the system Erlang > (/usr/lib/erlang/lib in my case) because the reltool doc clearly states > that "Applications may also be located outside root_dir ...". Does this > just not work? I'm not keen on installing my apps into the system Erlang > (as root) just so that I can build a release! You should be able to use lib_dirs to refer to your apps if they reside outside root_dir. But do observe that each lib_dir sould refer to a directory containing one sub directory for each app. It is the same directory structure as $ERL_TOP/lib/erlang/lib. > I've discovered that if I use reltool:create_target() to build a release > (and don't include any custom apps, so it works), I can't then use that > release as a development environment because running reltools fails on > all applications, even kernel. It seems to be that reltool doesn't > understand that things have been packaged up as .ez files because > manually extracting them all allows it to succeed. Is there a way to get > reltool to build a release that isn't zipped up, or to understand the > zipped files? Reltool should cope with sources in .ez files. I have not tried this for a long time, so it may be broken. > Should I instead be following the "target_system" module that's in the > "Creating a First Target System" section of the documentation? It seems > pretty odd to cut and paste a big chunk of example code out of the docs > to do a basic task... and it's also very slow because of the way it tars > everything up just to untar it again. This is, however, the only way > I've managed to get a release of my own app actually built. This should not be necessary. Reltool should be able to do that for you. > Should I just build .rel files by hand and basically re-implement > target_system.erl (but more the way I want it to work)? This should not be necessary. Reltool should be able to do that for you. It is hard to tell what it is that is causing your Reltool problems. It seems like you have understood how it is intended to work (despite the brief documentation). In order to dig deeper I would need see your indata to reltool:create_target() and look at the directory structure under your lib_dirs. Another possibility is that you take a look at how this little escript (search for --install) creates a target system with one app that is located outside root_dir: https://github.com/hawk/eflex/blob/master/bin/eflex You can download the app, build it and try to install it as a standalone tool. If that works it is perhaps possible to figure out what it is that differs with your original setup. Please, let me know how it works out for you. /H?kan From max.lapshin@REDACTED Mon Mar 28 17:16:02 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 28 Mar 2011 19:16:02 +0400 Subject: [erlang-questions] Re: [erlang-questions 82] Re: code:lib_dir and .ez archives In-Reply-To: <201103281506.p2SF6is9049367@pluto.hedeland.org> References: <AANLkTimG67+qub+6_ertuAQAxN+P-UP3qK5gDwNZ5Ve_@mail.gmail.com> <201103281506.p2SF6is9049367@pluto.hedeland.org> Message-ID: <AANLkTimWw8LUsvgp0Ow_deAW_nrgzqQSj0a+=Y4jF10p@mail.gmail.com> Maybe just add something like os:daemonize() ? main(Args) -> .... init stuff .... case lists:member("-d", Args) of true -> os:daemonize(); false -> launch_shell() end. ? From danielstritt@REDACTED Mon Mar 28 20:54:03 2011 From: danielstritt@REDACTED (Daniel) Date: Mon, 28 Mar 2011 13:54:03 -0500 Subject: [erlang-questions] Differences in Erlang shells in Windows Message-ID: <20110328145403.47OWY.216474.imail@eastrmwml32> Hello, I am using the Windows build of the latest Erlang. I am just curious as to what the difference in functionality is of the shells "erl.exe" and "werl.exe". I'm kind of new to this, and I want to make sure I am using the right one. Thanks, Daniel From antoine.koener@REDACTED Mon Mar 28 21:22:18 2011 From: antoine.koener@REDACTED (Koener Antoine) Date: Mon, 28 Mar 2011 21:22:18 +0200 Subject: [erlang-questions] Re: Differences in Erlang shells in Windows In-Reply-To: <20110328145403.47OWY.216474.imail@eastrmwml32> References: <20110328145403.47OWY.216474.imail@eastrmwml32> Message-ID: <45C0EEFF-934E-4555-AECB-69B5D2D69C23@gmail.com> > is of the shells "erl.exe" and "werl.exe". I'm kind of new to this, > and I want to make sure I am using the right one. Easy: erl.exe don't handle completion, (i.e. TAB is a \t) and runs in 'cmd.exe' werl.exe is autonomous, you can change the font, you have completion (TAB completes) erl.exe is needed to launch a service (see erlsrv) werl.exe is the prefered window shell. Finally experiment yourself, you'll find that werl is the only way to go on windows... On Mar 28, 2011, at 20:54 , Daniel wrote: > Hello, I am using the Windows build of the latest Erlang. I am just > curious as to what the difference in functionality is of the shells > "erl.exe" and "werl.exe". I'm kind of new to this, and I want to > make sure I am using the right one. > > Thanks, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From per.melin@REDACTED Mon Mar 28 22:31:34 2011 From: per.melin@REDACTED (Per Melin) Date: Mon, 28 Mar 2011 22:31:34 +0200 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <4D902590.8060409@m5net.com> References: <4D902590.8060409@m5net.com> Message-ID: <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> On Mon, Mar 28, 2011 at 8:07 AM, Sam Bobroff <sam@REDACTED> wrote: > I've been trying to work out how to "properly" build Erlang releases (on > a Linux system, to be deployed on another Linux system), and I'm getting > nowhere fast so I'm hoping someone here will have some useful pointers > or hints to help me out. I spent many, many hours in utter frustration before I got this working for the first time. I now use a ~400 line escript that I wrote that glues together reltool, systools and release_handler. What follows should be the minimal release possible with a custom boot file. I create a dummy app called foo that doesn't have any modules, only a foo.app. fanboy:~$ mkdir -p /tmp/minirel fanboy:~$ cd /tmp/minirel fanboy:minirel$ mkdir -p lib/foo/ebin fanboy:minirel$ cat > lib/foo/ebin/foo.app {application, foo, [{description, ""}, {vsn, "1.0.0"}, {modules, []}, {registered, []}, {included_applications, []}, {applications, [kernel,stdlib]}, {env, []}]}. fanboy:minirel$ mkdir rel fanboy:minirel$ erl Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> reltool:create_target([{sys, [{lib_dirs, ["lib"]}, {app, foo, [{incl_cond, include}]}, {incl_archive_filters, []}, {rel, "foo", "001", [foo, kernel, stdlib, sasl]}, {boot_rel, "foo"}]}], "rel"). ok 2> User switch command --> q fanboy:minirel$ ls -1F rel/ Install* bin/ erts-5.8.3/ lib/ misc/ releases/ usr/ fanboy:minirel$ ls rel/lib/foo-1.0.0/ ebin fanboy:minirel$ ls rel/releases/001/ foo.boot foo.rel foo.script --- And even when you get this far, it is just the first step on a painful journey. But once you get it working all the way it's a beautiful thing. (Since R14 I don't think you need the {app, foo, [{incl_cond, include}]} anymore if you list your app in the rel-tuple.) From moxford@REDACTED Mon Mar 28 22:44:46 2011 From: moxford@REDACTED (Mike Oxford) Date: Mon, 28 Mar 2011 13:44:46 -0700 Subject: [erlang-questions] Re: [erlang-questions 65] Erlang/OTP configure can not find wxWidgets In-Reply-To: <4D8FC063.3030203@gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> <4D8FC063.3030203@gmail.com> Message-ID: <AANLkTimuEcegqLO8X01bLUEbqd3KPgggVSyES+=2Jkrp@mail.gmail.com> <dream> May I suggest starting a "movement" away from wxWidgets and towards Qt? </dreaming> Or at least decoupling Erlang core from wxWidgets? Does anyone "in the know" have any ideas on how monstrous of a project that would be? Thanks! -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110328/023212fc/attachment.htm> From per@REDACTED Mon Mar 28 22:50:58 2011 From: per@REDACTED (Per Hedeland) Date: Mon, 28 Mar 2011 22:50:58 +0200 (CEST) Subject: [erlang-questions] Re: [erlang-questions 82] Re: code:lib_dir and .ez archives In-Reply-To: <AANLkTimWw8LUsvgp0Ow_deAW_nrgzqQSj0a+=Y4jF10p@mail.gmail.com> Message-ID: <201103282050.p2SKowFp055499@pluto.hedeland.org> Max Lapshin wrote: > >Maybe just add something like os:daemonize() ? Uhh, "just"? Do you have some suggestions for the implementation? I guess you're thinking of an analogue to the daemon(3) function that can commonly be found in the C library on *nix. This function calls fork(2), and subsequently the parent process exits, while execution continues in the child (which returns from the daemon() call). Obviously this can't be done in Erlang code, and while a (Unix-only, of course) BIF that does exactly the same thing as daemon(3) wouldn't be hard to implement per se, I wouldn't want to even begin to figure out what effects it would have on the Erlang VM at some arbitrary point in its execution. If it really is trivial to get it to work, you can "just" write a NIF... --Per From fred.hebert@REDACTED Mon Mar 28 23:05:31 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Mon, 28 Mar 2011 14:05:31 -0700 Subject: [erlang-questions] Re: [erlang-questions 15] Re: New Mailing list software In-Reply-To: <20110328073403.GA13069@erix.ericsson.se> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <56F659B0-8538-47C2-85EA-7DA768050683@erlang-solutions.com> <20110328073403.GA13069@erix.ericsson.se> Message-ID: <27B42F05-9831-4F0F-8D12-A13DDCBF8879@erlang-solutions.com> My bad, I got confused with the following (http://www.erlang.org/</div><div><h1 style="font-size:60px;">Powered by node.js</h1></div>) which doesn't impact the ML software, but the Erlang.org site. On 2011-03-28, at 00:34 AM, Raimo Niskanen wrote: > On Fri, Mar 25, 2011 at 02:22:50PM -0700, Fr?d?ric Trottier-H?bert wrote: >> No junk for me, although I worry about the XSS issue pointed earlier. > > Sorry, which XSS issue? > >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> >> >> On 2011-03-25, at 14:18 PM, Koener Antoine wrote: >> >>> Hi, >>> >>> Every single mail seems to be seen as Junk now in Apple Mail. >>> >>> Is this just me ? :) >>> >>> >>> PS: I know that there's many other imap clients out there :-) >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -- Fred H?bert http://www.erlang-solutions.com From fritchie@REDACTED Mon Mar 28 23:29:18 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 28 Mar 2011 16:29:18 -0500 Subject: [erlang-questions] Small update on cluster_info app's status Message-ID: <91598.1301347758@snookles.snookles.com> Hi, all. Erlang Factory was a fun thing. it was wonderful to see many old friends and meet new ones. I wanted to let everyone know about a correction that I added to the README file for the cluster_info application. It was part of my talk at Erlang Factory that gave a tour of all of the public source code repos (33 of them!) that Basho has hosted at GitHub. The slides for my talk are now available at the Erlang Factory web server, as are many others. Hopefully the rest will appear in the next week or so? http://www.erlang-factory.com/conference/SFBay2011/talks There's lots of good stuff to read there, especially with three tracks happening simultaneously and no way to clone myself to attend them all. The cluster_info part of my slides starts at about slide #37. Also, I'll cut-and-paste the text of the cluster_info slides below. -Scott --- snip --- snip --- snip --- snip --- snip --- snip --- snip --- Where is the code that does that? --------------------------------- For Basho-specific packaging reasons, the code that implements many of the basic info-generating functions are not contained in this source repo. I apologize for that. Please see the following source: https://github.com/basho/riak_core/blob/master/src/riak_core_cinfo_basic.erl This code is also licensed under the Apache Public License version 2. Please feel free to use as much or as little of it as you wish. Meanwhile, I will look into refactoring the code to see if it's practical to move it out of the `riak_core` source repo and into this one. --- snip --- snip --- snip --- snip --- snip --- snip --- snip --- cluster_info ------------ How many times have you needed more info about an Erlang system? * Memory usage, # of processes, RAM used, ETS table sizes, # of ports in use, ... * You need it for all nodes in the cluster. * You need it simple. Single-command simple. * dump_all_connected("/tmp/out.txt"). * All nodes' output -> one file * Includes all of the info bits mentioned earlier * ... and very easy to extend. % egrep '^==* ' /tmp/out.txt == Node: 'riak@REDACTED' = Generator name: Current time and date = Generator name: VM statistics = Generator name: erlang:memory() summary = Generator name: Top 50 process memory hogs = Generator name: Registered process names = Generator name: Registered process name via regs() = Generator name: Non-zero mailbox sizes = Generator name: Ports = Generator name: Applications = Generator name: Timer status = Generator name: ETS summary = Generator name: Nodes summary = Generator name: net_kernel summary = Generator name: inet_db summary = Generator name: Alarm summary = Generator name: Global summary = Generator name: erlang:system_info() summary = Generator name: Loaded modules [... output truncated ...] From sam@REDACTED Tue Mar 29 05:05:23 2011 From: sam@REDACTED (Sam Bobroff) Date: Tue, 29 Mar 2011 14:05:23 +1100 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <AANLkTikmH3+RFUu8g7tv5GDxYXeZZ6RUpe=HebMLh3W2@mail.gmail.com> References: <4D902590.8060409@m5net.com> <AANLkTikmH3+RFUu8g7tv5GDxYXeZZ6RUpe=HebMLh3W2@mail.gmail.com> Message-ID: <4D914C73.2060400@m5net.com> On 29/03/11 02:10, H?kan Mattsson wrote: > On Mon, Mar 28, 2011 at 8:07 AM, Sam Bobroff <sam@REDACTED> wrote: [snip] >> If so, how do I get reltool:create_target() to include my apps? I've >> tried using lib_dirs and -pa in various combinations but I always get >> "Release NAME uses non existing application APP". I haven't yet tried >> installing my application directly to the system Erlang >> (/usr/lib/erlang/lib in my case) because the reltool doc clearly states >> that "Applications may also be located outside root_dir ...". Does this >> just not work? I'm not keen on installing my apps into the system Erlang >> (as root) just so that I can build a release! > You should be able to use lib_dirs to refer to your apps if they reside > outside root_dir. But do observe that each lib_dir sould refer to a > directory containing one sub directory for each app. It is the same > directory structure as $ERL_TOP/lib/erlang/lib. [snip] Ah! Thank you so much H?kan, with the correct lib_dirs setting I am able to create releases and have them start my apps. I knew it would be something simple :-) (The same adjustment allows rebar to work as well.) Peace, Sam. From bo.zhang86@REDACTED Tue Mar 29 05:45:06 2011 From: bo.zhang86@REDACTED (zhangbo) Date: Tue, 29 Mar 2011 11:45:06 +0800 Subject: [erlang-questions] about a tcp question in erlang In-Reply-To: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> Message-ID: <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> Hi, Code: {ok, Listen} = gen_tcp:listen(80, [{packet,0}, binary, {active, true}]) Loop: receive {tcp, S, Bin} -> ... I want to know, when a client send a message("helloworld") to server, then send another message("helloworld2"). How many messages will the server receive? (I guess 2.) If the client send very fast, or the message is big enough, how the situation again? When receive {tcp, S, Bin}, Bin stands for a message or maybe just a fragment of message? If the client send "hello, server", Bin maybe "hello, server", but If the message is very big, what will Bin be? and how many times will the loop receive? From jack@REDACTED Tue Mar 29 06:24:19 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 28 Mar 2011 22:24:19 -0600 Subject: [erlang-questions] Re: about a tcp question in erlang In-Reply-To: <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> Message-ID: <AANLkTimjLf-7NOoYxF8p=0ASA_ym6De9tehEHS_AmCO-@mail.gmail.com> > I want to know, when a client send a message("helloworld") to server, then send another message("helloworld2"). How many messages will the server receive? (I guess 2.) ?If the client send very fast, or the message is big enough, how the situation again? With {active, true} the client can overflow your receiver. If you want flow control, you can use {active, once} and then reset the socket option after every receive, which will allow another message to come through. From what I've seen, {active, once} is the standard setup for TCP sockets in Erlang. jack. From bo.zhang86@REDACTED Tue Mar 29 07:32:59 2011 From: bo.zhang86@REDACTED (zhangbo) Date: Tue, 29 Mar 2011 13:32:59 +0800 Subject: [erlang-questions] Re: about a tcp question in erlang In-Reply-To: <AANLkTimjLf-7NOoYxF8p=0ASA_ym6De9tehEHS_AmCO-@mail.gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> <AANLkTimjLf-7NOoYxF8p=0ASA_ym6De9tehEHS_AmCO-@mail.gmail.com> Message-ID: <8DC83249-AA87-47F3-9CA2-AB42FB134673@gmail.com> Thank you. How about the messages that the server receive? For example, the client send a big message A(eg. 65536 bytes), what will happen when receive the message? Maybe I should ask question like this: the client send a message(maybe it's big), in server loop, how many messages it will receive? {ok, Listen} = gen_tcp:listen(80, [{packet,0}, binary, {active, true}]) Loop: receive {tcp, S, Bin} -> ... ? 2011-3-29???12:24? Jack Moffitt ??? > m what I've seen, {active, on -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/c1fcfd90/attachment.htm> From sam@REDACTED Tue Mar 29 07:58:54 2011 From: sam@REDACTED (Sam Bobroff) Date: Tue, 29 Mar 2011 16:58:54 +1100 Subject: [erlang-questions] Upgrading a gen_tcp acceptor process Message-ID: <4D91751E.5040508@m5net.com> Hello again Erlangers, I've been teaching myself about code reloading (or "hot upgrades") and I've hit problem with my very simple test application. The problem occurs in a server that accepts TCP connections and is: (1) To do it's work, the process must at some point call gen_tcp:accept() to accept new connections, which blocks. (2) To be upgraded, the process must handle system messages in a receive loop. I can't see a way to reconcile (1) and (2) without polling constantly by adding a timeout to gen_tcp:accept(), and this seems to be a horrible solution. Given that this is probably a very common situation, how am I supposed to handle it? (I don't want to confuse things, but it seems like the interface to gen_tcp:accept() causes another problem: It can't be put in a gen_server for the same reason. It seems "obvious" that there should be a message passing interface to it but I don't see an EEP for it; is there some reason why it couldn't work that way?) Thanks in advance for any comments or pointers! Peace, Sam. From t.dt.aanderson@REDACTED Tue Mar 29 08:34:19 2011 From: t.dt.aanderson@REDACTED (Thomas Anderson) Date: Tue, 29 Mar 2011 14:34:19 +0800 Subject: [erlang-questions] Internal mechanism of monitor Message-ID: <AANLkTimBnPPmFqdFjmZ+cuNtbkF1k455yxE+BmC6qgve@mail.gmail.com> I read some articles talking about erlang and am caught attention by erlang's intrinsically fault tolerance features. In the paper `Making reliable distributed system in the presence of software error,' it explains error handling with e.g links and monitor. But it only refers to function such as link(), spwan_link(), monitor(), etc., and some basic illustration about these two functions. After searching on erlang.org as well as the internet I do not find document or threads discussing its internal mechanism. For instance, is it failure detector? Or does it collect metrics data within a process? I also try to read the code obtained from erlang.org, it seems erlang.erl is the right place to start from; but because my limited knowledge to erlang (I am completely new to function programming and erlang), I can not understand that very well. So my question - is there any document or threads discussing internal mechanism about such issue (in conceptual) about erlang? Or place where there may contain such resource? I appreciate any help. From bengt.kleberg@REDACTED Tue Mar 29 08:41:11 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 29 Mar 2011 08:41:11 +0200 Subject: [erlang-questions] Re: [erlang-questions 65] Erlang/OTP configure can not find wxWidgets In-Reply-To: <AANLkTimuEcegqLO8X01bLUEbqd3KPgggVSyES+=2Jkrp@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <AANLkTiksQaKd58FmUjfWrZpGv7CtB9+k7P+L8GFGPHkZ@mail.gmail.com> <AANLkTima81177aZnDb=fs57R5yHp-+-P7jdrTx+UoyYd@mail.gmail.com> <AANLkTinvuJdx9N=-Ph2Oz5+NsyX05k1uuov4GiKUKOSe@mail.gmail.com> <4D8FC063.3030203@gmail.com> <AANLkTimuEcegqLO8X01bLUEbqd3KPgggVSyES+=2Jkrp@mail.gmail.com> Message-ID: <1301380871.5251.23.camel@seasc1137> Greetings, Is there a definition of "Erlang core"? bengt On Mon, 2011-03-28 at 22:44 +0200, Mike Oxford wrote: > <dream> > May I suggest starting a "movement" away from wxWidgets and towards > Qt? > </dreaming> > > > Or at least decoupling Erlang core from wxWidgets? > > > Does anyone "in the know" have any ideas on how monstrous of a project > that would be? > > > Thanks! > > > -mox > > From bengt.kleberg@REDACTED Tue Mar 29 08:48:01 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 29 Mar 2011 08:48:01 +0200 Subject: [erlang-questions] Re: Upgrading a gen_tcp acceptor process In-Reply-To: <4D91751E.5040508@m5net.com> References: <4D91751E.5040508@m5net.com> Message-ID: <1301381281.5251.27.camel@seasc1137> Greetings, This is only theoretical speculations. How about a very simple process that does nothing but 1) gen_tcp:accept/1 2) send the new socket to the gen_server 3) repeat This process you kill and restart, if needed, from the gen_server during upgrade. bengt On Tue, 2011-03-29 at 07:58 +0200, Sam Bobroff wrote: > Hello again Erlangers, > > I've been teaching myself about code reloading (or "hot upgrades") and > I've hit problem with my very simple test application. > > The problem occurs in a server that accepts TCP connections and is: > > (1) To do it's work, the process must at some point call > gen_tcp:accept() to accept new connections, which blocks. > (2) To be upgraded, the process must handle system messages in a receive > loop. > > I can't see a way to reconcile (1) and (2) without polling constantly by > adding a timeout to gen_tcp:accept(), and this seems to be a horrible > solution. Given that this is probably a very common situation, how am I > supposed to handle it? > > (I don't want to confuse things, but it seems like the interface to > gen_tcp:accept() causes another problem: It can't be put in a gen_server > for the same reason. It seems "obvious" that there should be a message > passing interface to it but I don't see an EEP for it; is there some > reason why it couldn't work that way?) > > Thanks in advance for any comments or pointers! > > Peace, > Sam. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From nox@REDACTED Tue Mar 29 10:08:57 2011 From: nox@REDACTED (Anthony Ramine) Date: Tue, 29 Mar 2011 10:08:57 +0200 Subject: [erlang-questions] Re: about a tcp question in erlang In-Reply-To: <8DC83249-AA87-47F3-9CA2-AB42FB134673@gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> <AANLkTimjLf-7NOoYxF8p=0ASA_ym6De9tehEHS_AmCO-@mail.gmail.com> <8DC83249-AA87-47F3-9CA2-AB42FB134673@gmail.com> Message-ID: <9285FA3B-BBF6-4FE0-BE1B-5F2DB74408C6@dev-extend.eu> You don't know and have no way to know. Le 29 mars 2011 ? 07:32, zhangbo a ?crit : > Thank you. How about the messages that the server receive? For example, the client send a big message A(eg. 65536 bytes), what will happen when receive the message? > Maybe I should ask question like this: the client send a message(maybe it's big), in server loop, how many messages it will receive? > > {ok, Listen} = gen_tcp:listen(80, [{packet,0}, > binary, > {active, true}]) > > Loop: > receive > {tcp, S, Bin} -> > ... > -- Anthony Ramine Dev:Extend http://dev-extend.eu From ekeroth.christoffer@REDACTED Tue Mar 29 10:33:37 2011 From: ekeroth.christoffer@REDACTED (JohnyTex) Date: Tue, 29 Mar 2011 01:33:37 -0700 (PDT) Subject: [erlang-questions] Newbie Erlang programmer looking for feedback! :) Message-ID: <c4415989-af65-4fb5-bde1-ff93738e9155@f2g2000yqf.googlegroups.com> Hello everybody! I'm new to Erlang and pretty new to functional programming in general. I've been having a really good time with Erlang so far (even though Erlang's punctuation has had me trip up a few times ;)), but I'd really love it if I could get some feedback on my code from more experienced Erlang programmers. My code seems to work alright, but I'm sure you guys could offer a lot of advice for improvement! :) Anyway, here's my solution to the second problem from Project Euler (http://projecteuler.net/index.php?section=problems&id=2). The problem is to find the sum of all the even fibonacci numbers below four million; I thought this could be a good exercise since it would let me try my hand at implementing lazy sequences and working with lists. I split up the program into two modules. The first one is for handling lazy sequences, since I thought that would be a good way to find our list of primes: -module(seqs). -export([takewhile/2, take/2]). %% Recursively pick elements from a lazy sequence while Pred(H) is true takewhile(Pred, [H|T]) -> case Pred(H) of true -> [H|takewhile(Pred, T())]; false -> [] end. %% Take a certain number of elements from a lazy sequence %% A non-tail recursive version take(0, _) -> []; take(Number, [H|T]) -> [H|take(Number - 1, T())]. The second module actually solves the problem: -module(euler002). -import(seqs, [takewhile/2]). -export([lazyfib/0, solve/0]). %% Sums the numbers in a list (for practice's sake) sum([]) -> 0; sum([H|T]) -> H + sum(T). %% Practicing some list comprehensions as well! filter(P, Xs) -> [ X || X <- Xs, P(X) ]. %% Lazy sequence that generates fibonacci numbers lazyfib(A, B) -> [A | fun () -> lazyfib(B, A + B) end]. lazyfib() -> lazyfib(0, 1). %% Generate all fibonacci terms that are less than 4 million and sum the %% even terms solve() -> Fibs = seqs:takewhile(fun (X) -> X < 4000000 end, lazyfib()), sum(filter(fun (X) -> X rem 2 =:= 0 end, Fibs)). Thanks in advance, and please tell me if this is not the appropriate forum for this kind of question! :) From jwest@REDACTED Tue Mar 29 10:36:35 2011 From: jwest@REDACTED (Jordan West) Date: Tue, 29 Mar 2011 01:36:35 -0700 Subject: [erlang-questions] Common Test Question Message-ID: <BANLkTi=g8rsUgv-tZUw1bALiVd5BWPViaQ@mail.gmail.com> I've recently been using common_test on several new projects and am really liking it. I ran across something today I found odd, however, and wanted to know if it was the expected behavior. I was expecting that if I have a group with no tests that group should, essentially, be ignored. This seems to be the case if the group is contained in another but not if the group is an element in the list returned by all/0. In the latter case the suite errors out with "Error in suite detected: 'Invalid reference to group [GROUP NAME] in [SUITE NAME]:all/0'". Is this intentional and if so, why are sub-groups treated differently? Is there a reason its not a warning instead? For the most part I'm just curious. I do like setting up my groups sometimes ahead of writing tests and ran into this when I tried to leave a top-level group empty for a short while. A minimal code example is below. I'm using R14B02 on OS-X with common_test 1.5.3 (I installed today off the Erlang Factory USB stick, thanks for a great conference everyone). Oh, and this is my first message to the list. Hello, all. %% a.erl -module(a). -compile(export_all). my_fun() -> 1. %% a_SUITE.erl -module(a_SUITE). -compile(export_all). groups() -> [{group_1, [shuffle], [test_1, {group, group_3}]}, {group_2, [shuffle], []}, % this empty test list causes issues if empty {group_3, [],[]}]. % this empty test list seems to be able to be empty all() -> [ {group, group_1}, {group, group_2} %% If this is remove the suite won't error out ]. test_1(_) -> 1 = a:my_fun(). >> run_test -suite a_SUITE.erl ... Error in suite detected: 'Invalid reference to group group_2 in a_SUITE:all/0' Jordan West -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/9c916d85/attachment.htm> From hm@REDACTED Tue Mar 29 10:48:57 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 29 Mar 2011 10:48:57 +0200 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> References: <4D902590.8060409@m5net.com> <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> Message-ID: <AANLkTim0H4KwjKVe9mLzAzAhmdMxr7jiueea1uGoiB3k@mail.gmail.com> On Mon, Mar 28, 2011 at 10:31 PM, Per Melin <per.melin@REDACTED> wrote: > (Since R14 I don't think you need the {app, foo, [{incl_cond, > include}]} anymore if you list your app in the rel-tuple.) Yes, this simplification was introduced in reltool 0.5.4 (april 2010). In order to include an application in the target system it does now suffice to list it in a release (rel-tuple). All dependent applications will also be included. It is only when you want to process an application differently than the others that you need to explicitly add it as an application (app-tuple). Further you do not need to bother about kernel and stdlib. They are automatically included in all releases as they are mandatory. /H?kan From pekadan@REDACTED Tue Mar 29 10:56:49 2011 From: pekadan@REDACTED (Peter Andersson) Date: Tue, 29 Mar 2011 10:56:49 +0200 Subject: [erlang-questions] Re: Common Test Question In-Reply-To: <BANLkTi=g8rsUgv-tZUw1bALiVd5BWPViaQ@mail.gmail.com> References: <BANLkTi=g8rsUgv-tZUw1bALiVd5BWPViaQ@mail.gmail.com> Message-ID: <AANLkTimiHfYK8g8AMFo9PgJJfzG2CC90X0Af+ARrV3_d@mail.gmail.com> Hi Jordan, I learned about this problem just the other day. I consider it a bug really. An empty group should be ignored, or possibly, the config functions for the group should be executed (even if there's no test case to configure for). I have written a ticket on it and a fix will come shortly. Thanks for reporting! Best regards, Peter Ericsson AB, Erlang/OTP 2011/3/29 Jordan West <jwest@REDACTED>: > I've recently been using common_test on several new projects and am really > liking it. I ran across something today I found odd, however, and wanted to > know if it was the expected behavior. I was expecting that if I have a > group with no tests that group should, essentially, be ignored. This seems > to be the case if the group is contained in another but not if the group is > an element in the list returned by all/0. In the latter case the suite > errors out with "Error in suite detected: 'Invalid reference to group [GROUP > NAME] in [SUITE NAME]:all/0'". > Is this intentional and if so, why are sub-groups treated differently? Is > there a reason its not a warning instead? For the most part I'm just > curious. I do like setting up my groups sometimes ahead of writing tests and > ran into this when I tried to leave a top-level group empty for a short > while. > A minimal code example is below. I'm using R14B02 on OS-X with common_test > 1.5.3 (I installed today off the Erlang Factory USB stick, thanks for a > great conference everyone). Oh, and this is my first message to the list. > Hello, all. > %% a.erl > -module(a). > -compile(export_all). > my_fun() -> > 1. > %% a_SUITE.erl > -module(a_SUITE). > -compile(export_all). > groups() -> > [{group_1, > [shuffle], > [test_1, {group, group_3}]}, > {group_2, > [shuffle], > []}, % this empty test list causes issues if empty > {group_3, > [],[]}]. % this empty test list seems to be able to be empty > all() -> > [ > {group, group_1}, > {group, group_2} %% If this is remove the suite won't error out > ]. > test_1(_) -> > 1 = a:my_fun(). >>> run_test -suite a_SUITE.erl > ... > Error in suite detected: 'Invalid reference to group group_2 in > a_SUITE:all/0' > Jordan West > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From attila.r.nohl@REDACTED Tue Mar 29 11:08:49 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 29 Mar 2011 11:08:49 +0200 Subject: [erlang-questions] Re: Internal mechanism of monitor In-Reply-To: <AANLkTimBnPPmFqdFjmZ+cuNtbkF1k455yxE+BmC6qgve@mail.gmail.com> References: <AANLkTimBnPPmFqdFjmZ+cuNtbkF1k455yxE+BmC6qgve@mail.gmail.com> Message-ID: <AANLkTim7Yz+1NO6tRdQH8hbrp4E2+nre4a8Qwpqu29kQ@mail.gmail.com> 2011/3/29, Thomas Anderson <t.dt.aanderson@REDACTED>: > I read some articles talking about erlang and am caught attention by > erlang's intrinsically fault tolerance features. > > In the paper `Making reliable distributed system in the presence of > software error,' it explains error handling with e.g links and > monitor. But it only refers to function such as link(), spwan_link(), > monitor(), etc., and some basic illustration about these two > functions. After searching on erlang.org as well as the internet I do > not find document or threads discussing its internal mechanism. For > instance, is it failure detector? The documentation says that "erlang:monitor(Type, Item) -> MonitorRef ... The calling process starts monitoring Item which is an object of type Type. Currently only processes can be monitored, i.e. the only allowed Type is process ... A 'DOWN' message will be sent to the monitoring process if Item dies, if Item does not exist, or if the connection is lost to the node which Item resides on." So it's a failure detector. From apauley@REDACTED Tue Mar 29 12:02:11 2011 From: apauley@REDACTED (Andreas Pauley) Date: Tue, 29 Mar 2011 12:02:11 +0200 Subject: [erlang-questions] Re: [erlang-questions 39] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTimLHaUJr0d5qmX53-jvBNKdV+rZH9SiJCwJEd3O@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> <AANLkTimLHaUJr0d5qmX53-jvBNKdV+rZH9SiJCwJEd3O@mail.gmail.com> Message-ID: <AANLkTikh7VakuE+anS3GnbUyOjWGq10sc04StVN9ptTb@mail.gmail.com> Unfortunately it's not that easy this time. The major reduction in eliminations happens because is_unit_solved/2 always returns true after the proposed change. If you open any of the .out files (eg hardest.out) you'll see that the puzzles aren't really solved. Luckily the unit tests pointed this out. But all is not lost, I still got a simplification out of the deal. We still need to check the non-unique length of UnitValues in that function, but the second clause does not need to use sets anymore: https://github.com/apauley/sudoku-in-erlang/commit/f092e55df292d1f12ff8406ce60bb435e7a83667 On Mon, Mar 28, 2011 at 4:39 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > Hi Andreas, > For some reason, lists:usort is your friend again! > Looking into the code deeper, one line can give u huge boost! > You compare the unit values with digits with > ??(length(UnitValues) == 9) > ?? ?and (gb_sets:from_list(UnitValues) == gb_sets:from_list(digits())). > I understand (correct me if i'm wrong) u check first the length to make sure > it's 9 digits then u convert to sets to make sure they r unique and compare > them. But if you think about it, if the unique list of values is of length > 9, u don't need more comparisons! > so u can actually just do > (length(lists:usort(UnitValues)==9) > before the change: > sudoku:print_results("top95.txt"). > Solved 95 of 95 puzzles from top95.txt in 1.937175 secs (49.04 Hz) > ??(210794 total eliminations, avg 2218.88, median 1536, max 10664, min 648). > after the change : > sudoku:print_results("top95.txt"). > Solved 95 of 95 puzzles from top95.txt in 0.382337 secs (248.47 Hz) > ??(45002 total eliminations, avg 473.71, median 479, max 518, min 427). > > On Mon, Mar 28, 2011 at 3:32 PM, Andreas Pauley <apauley@REDACTED> wrote: >> >> Hi Tristan, >> >> I tried the code in your repo, but it wasn't significantly faster on my >> machine. >> >> It was interesting though to go through your refactorings. >> I see you changed the signature for solve so that the empty puzzle and >> the list of square names are passed in from the start. >> Maybe I should try this approach in conjunction with the conversion to >> atoms... >> >> Regards, >> Andreas >> >> On Sat, Mar 26, 2011 at 10:56 PM, Tristan Sloughter >> <tristan.sloughter@REDACTED> wrote: >> > With a few refactorings (https://github.com/tsloughter/sudoku-in-erlang >> > -- >> > note I used ewl_plists just to make the code simpler: >> > https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile >> > with >> > native, hipe o3 it beats python's: >> > $ ./sudoku.py >> > All tests pass. >> > Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) >> > ??(33059 total eliminations, avg 661.00, median 648, max 830, min 648). >> > Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) >> > ??(221997 total eliminations, avg 2336.00, median 1492, max 11512, min >> > 648). >> > Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) >> > ??(9436 total eliminations, avg 857.00, median 817, max 1198, min 648). >> > $ ./sudoku >> > All tests passed :-) >> > Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) >> > ??(92538 total eliminations, avg 1850.76, median 1811, max 2628, min >> > 1767). >> > Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) >> > ??(901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> > 1792). >> > Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) >> > ??(33653 total eliminations, avg 3059.36, median 3023, max 5346, min >> > 1786). >> > I ran a number of times, those are pretty much what it always comes out >> > to. >> > But I haven't really tried a full refactoring, I think a number of >> > things >> > can be reduced to less iterations. >> > >> > Tristan >> > On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> >> > wrote: >> >> >> >> BTW, beside looking into reducing eleminations (which requires deep >> >> dive), >> >> if u do some time profiling using eprof u will find that most of the >> >> time >> >> spent?actually?in the cross functions >> >> sudoku:'-cross/2-lc$^1/1-1-'/4 ? ? ? ? ? ? ? ? ? ? ? 96558660 ?37.46 >> >> ?35024918 ?[ ? ? ?0.36] >> >> >> >> On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> >> >> wrote: >> >>> >> >>> actually instead of doing >> >>> >> >>> ????NonUniquePeers = shallow_flatten([S || S <- units(Square)]), >> >>> >> >>> >> >>> ????PeerSet = sets:from_list(NonUniquePeers), >> >>> >> >>> >> >>> ????PeersWithSelf = sets:to_list(PeerSet), >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> can't u just do >> >>> PeersWithSelf = lists:usort(NonUniquePeers). >> >>> >> >>> >> >>> ? >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED> >> >>> wrote: >> >>>> >> >>>> Good call....gb_sets should be faster. Compiled to native it runs >> >>>> faster >> >>>> still >> >>>> >> >>>> 9> sudoku:print_results("top95.txt", "\n"). >> >>>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) >> >>>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, min >> >>>> 1792). >> >>>> ok >> >>>> >> >>>> ________________________________________ >> >>>> From: Ahmed Omar [spawn.think@REDACTED] >> >>>> Sent: Saturday, March 26, 2011 3:08 PM >> >>>> To: Andreas Pauley >> >>>> Cc: Evans, Matthew; erlang-questions@REDACTED >> >>>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang >> >>>> compared to Python >> >>>> >> >>>> using gb_sets instead of sets could decrease eliminations a bit and >> >>>> give >> >>>> u some boost. However, i didn't dive deeper into the code or the >> >>>> algorithm. >> >>>> >> >>>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley >> >>>> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: >> >>>> Thanks, I've changed the compile options and this definitely makes it >> >>>> somewhat faster: >> >>>> >> >>>> >> >>>> https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb >> >>>> >> >>>> Strangely the native flag is not documented here: >> >>>> http://www.erlang.org/doc/man/compile.html#file-2 >> >>>> >> >>>> The more interesting improvement would be to decrease the number of >> >>>> eliminations performed, but for that I'll have to go deep into the >> >>>> code :-) >> >>>> >> >>>> >> >>>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew >> >>>> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: >> >>>> > Without going deep into the code, one thing to try is compile with >> >>>> > the >> >>>> > native flag (running the tests from the VM shell here): >> >>>> > >> >>>> > Without native set: >> >>>> > 18> c(sudoku). >> >>>> > {ok,sudoku} >> >>>> > 19> ?sudoku:print_results("top95.txt", "\n"). >> >>>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 Hz) >> >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, >> >>>> > min >> >>>> > 1797). >> >>>> > ok >> >>>> > >> >>>> > >> >>>> > With native set: >> >>>> > 20> c(sudoku,[native]). >> >>>> > {ok,sudoku} >> >>>> > 21> ?sudoku:print_results("top95.txt", "\n"). >> >>>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 Hz) >> >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, >> >>>> > min >> >>>> > 1797). >> >>>> > >> >>>> > >> >>>> > >> >>>> > -----Original Message----- >> >>>> > From: >> >>>> > >> >>>> > erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED> >> >>>> > >> >>>> > [mailto:erlang-questions-bounces@REDACTED<mailto:erlang-questions-bounces@REDACTED>] >> >>>> > On Behalf Of Andreas Pauley >> >>>> > Sent: Friday, March 25, 2011 8:15 AM >> >>>> > To: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> >>>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared to >> >>>> > Python >> >>>> > >> >>>> > Hi all, >> >>>> > >> >>>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku >> >>>> > solver into Erlang: >> >>>> > https://github.com/apauley/sudoku-in-erlang >> >>>> > >> >>>> > For comparison, my slightly modified version of Norvig's Python >> >>>> > code >> >>>> > can be found here: >> >>>> > https://github.com/apauley/sudoku-by-norvig >> >>>> > >> >>>> > I like the pattern matching, it was fun to do the entire solution >> >>>> > without a single if or case statement :-) >> >>>> > Something that bothers me though is that the Python solution seems >> >>>> > to >> >>>> > be faster, even after I've made the Erlang solution use multiple >> >>>> > processors. >> >>>> > >> >>>> > In order to compare the two solutions I counted the number of >> >>>> > eliminations each performed. >> >>>> > The eliminate function is the core of the solution, for example >> >>>> > assigning a single value to a square is implemented as the >> >>>> > elimination >> >>>> > of all other values. >> >>>> > >> >>>> > With the Erlang implementation I get: >> >>>> > sudoku-in-erlang$ ./sudoku >> >>>> > All tests passed :-) >> >>>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 Hz) >> >>>> > ?(93403 total eliminations, avg 1868.06, median 1810, max 2517, min >> >>>> > 1770). >> >>>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 Hz) >> >>>> > ?(922678 total eliminations, avg 9712.40, median 6596, max 55370, >> >>>> > min >> >>>> > 1797). >> >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 >> >>>> > Hz) >> >>>> > ?(32339 total eliminations, avg 2939.91, median 2894, max 4779, min >> >>>> > 1781). >> >>>> > >> >>>> > And with the Python implementation: >> >>>> > sudoku-by-norvig$ ./sudoku.py >> >>>> > All tests pass. >> >>>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 Hz) >> >>>> > ?(33059 total eliminations, avg 661.00, median 648, max 830, min >> >>>> > 648). >> >>>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 Hz) >> >>>> > ?(221997 total eliminations, avg 2336.00, median 1492, max 11512, >> >>>> > min >> >>>> > 648). >> >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 >> >>>> > Hz) >> >>>> > ?(9436 total eliminations, avg 857.00, median 817, max 1198, min >> >>>> > 648). >> >>>> > >> >>>> > So according to the stats above, the Python solution performs less >> >>>> > computations when given exactly the same input. >> >>>> > The Erlang code is as close to the Python as I could make it, I've >> >>>> > done more or less a direct translation of the algorithms used. >> >>>> > >> >>>> > I suspect that there are some lazy evaluation happening in the >> >>>> > Python >> >>>> > version, possibly generators, although I haven't pinpointed it yet. >> >>>> > >> >>>> > How can I improve my Erlang code in this solution? >> >>>> > >> >>>> > Kind regards, >> >>>> > Andreas >> >>>> > _______________________________________________ >> >>>> > erlang-questions mailing list >> >>>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> >>>> > http://erlang.org/mailman/listinfo/erlang-questions >> >>>> > >> >>>> _______________________________________________ >> >>>> erlang-questions mailing list >> >>>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> >> >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Best Regards, >> >>>> - Ahmed Omar >> >>>> http://nl.linkedin.com/in/adiaa >> >>>> Follow me on twitter >> >>>> @spawn_think<http://twitter.com/#!/spawn_think> >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Best Regards, >> >>> - Ahmed Omar >> >>> http://nl.linkedin.com/in/adiaa >> >>> Follow me on twitter >> >>> @spawn_think >> >> >> >> >> >> >> >> -- >> >> Best Regards, >> >> - Ahmed Omar >> >> http://nl.linkedin.com/in/adiaa >> >> Follow me on twitter >> >> @spawn_think >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > >> > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > From emile@REDACTED Tue Mar 29 12:12:37 2011 From: emile@REDACTED (Emile Joubert) Date: Tue, 29 Mar 2011 11:12:37 +0100 Subject: [erlang-questions] crypto on 64bit windows? Message-ID: <4D91B095.3040600@rabbitmq.com> Hi, Should the crypto module work on 64bit windows? It works fine on 32bit Windows XP, but on 64bit XP and 64bit Windows7 I get the following error: ---------- Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] Eshell V5.8.3 (abort with ^G) 1> crypto:start(). =ERROR REPORT==== 29-Mar-2011::11:05:13 === Unable to load crypto library. Failed with error: "load_failed, Failed to load NIF library c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The specified module could not be found.'" OpenSSL might not be installed on this system. =ERROR REPORT==== 29-Mar-2011::11:05:13 === The on_load function for module crypto returned {error, {load_failed, "Failed to load NIF library c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The specified module could not be found.'"}} ** exception error: undefined function crypto:start/0 ---------- C:\Program Files (x86)\erl5.8.3\lib\crypto-2.0.2.1\priv\lib does contain crypto.dll. Installing a 64bit version of OpenSSL for Windows doesn't help. The crypto documentation refers to Win32 - should I infer that crypto will not work on 64bit Windows operating systems? Regards Emile From per.melin@REDACTED Tue Mar 29 12:15:15 2011 From: per.melin@REDACTED (Per Melin) Date: Tue, 29 Mar 2011 12:15:15 +0200 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <AANLkTim0H4KwjKVe9mLzAzAhmdMxr7jiueea1uGoiB3k@mail.gmail.com> References: <4D902590.8060409@m5net.com> <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> <AANLkTim0H4KwjKVe9mLzAzAhmdMxr7jiueea1uGoiB3k@mail.gmail.com> Message-ID: <AANLkTinZw007nBx4zxVhcypSD3p9nV3MKcV=rUKrVCqN@mail.gmail.com> 2011/3/29 H?kan Mattsson <hm@REDACTED>: > On Mon, Mar 28, 2011 at 10:31 PM, Per Melin <per.melin@REDACTED> wrote: > >> (Since R14 I don't think you need the {app, foo, [{incl_cond, >> include}]} anymore if you list your app in the rel-tuple.) > > Yes, this simplification was introduced in reltool 0.5.4 (april 2010). In order > to include an application in the target system it does now suffice to > list it in a > release (rel-tuple). All dependent applications will also be included. > It is only > when you want to process an application differently than the others that you > need to explicitly add it as an application (app-tuple). Further you do not > need to bother about kernel and stdlib. They are automatically included in > all releases as they are mandatory. While we're on the subject, here are a couple of minor gotchas that could possibly be improved in future versions of reltool: 1. Where the .rel is placed. A release named "foo" with version "001" created with reltool:create_target/2 in target directory "foo_001" will get this .rel file: foo_001/releases/001/foo.rel But that is not what release_handler expects. So you need to move and possibly rename the file to: foo_001/foo_001.rel 2. start.boot. For the same release you get a boot file called foo.boot. But release_handler will barf unless there is a start.boot (which I believe is really irrelevant unless you run in embedded mode with the default scripts). So maybe when you include the option {boot_rel, "foo"} that could mean that foo.boot will instead be called start.boot? From spawn.think@REDACTED Tue Mar 29 12:31:45 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 29 Mar 2011 12:31:45 +0200 Subject: [erlang-questions] Re: [erlang-questions 39] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTikh7VakuE+anS3GnbUyOjWGq10sc04StVN9ptTb@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <AANLkTik371id2S_KYf7_c+6CgEqqBf1LJ5KDwU6Taqfh@mail.gmail.com> <AANLkTin89fNuWOKEdag+_sL=+Up9DRq8Z4_zJAnPSday@mail.gmail.com> <AANLkTimLHaUJr0d5qmX53-jvBNKdV+rZH9SiJCwJEd3O@mail.gmail.com> <AANLkTikh7VakuE+anS3GnbUyOjWGq10sc04StVN9ptTb@mail.gmail.com> Message-ID: <AANLkTikZb5BJ4d=bBZ=43GRjePFcZkqbq4t51QrpqCgd@mail.gmail.com> Yes u r correct :) sorry i missed that point, i didn't dig enough :) On Tue, Mar 29, 2011 at 12:02 PM, Andreas Pauley <apauley@REDACTED> wrote: > Unfortunately it's not that easy this time. > The major reduction in eliminations happens because is_unit_solved/2 > always returns true after the proposed change. > If you open any of the .out files (eg hardest.out) you'll see that the > puzzles aren't really solved. > > Luckily the unit tests pointed this out. > > But all is not lost, I still got a simplification out of the deal. > We still need to check the non-unique length of UnitValues in that > function, but the second clause does not need to use sets anymore: > > https://github.com/apauley/sudoku-in-erlang/commit/f092e55df292d1f12ff8406ce60bb435e7a83667 > > > On Mon, Mar 28, 2011 at 4:39 PM, Ahmed Omar <spawn.think@REDACTED> wrote: > > Hi Andreas, > > For some reason, lists:usort is your friend again! > > Looking into the code deeper, one line can give u huge boost! > > You compare the unit values with digits with > > (length(UnitValues) == 9) > > and (gb_sets:from_list(UnitValues) == gb_sets:from_list(digits())). > > I understand (correct me if i'm wrong) u check first the length to make > sure > > it's 9 digits then u convert to sets to make sure they r unique and > compare > > them. But if you think about it, if the unique list of values is of > length > > 9, u don't need more comparisons! > > so u can actually just do > > (length(lists:usort(UnitValues)==9) > > before the change: > > sudoku:print_results("top95.txt"). > > Solved 95 of 95 puzzles from top95.txt in 1.937175 secs (49.04 Hz) > > (210794 total eliminations, avg 2218.88, median 1536, max 10664, min > 648). > > after the change : > > sudoku:print_results("top95.txt"). > > Solved 95 of 95 puzzles from top95.txt in 0.382337 secs (248.47 Hz) > > (45002 total eliminations, avg 473.71, median 479, max 518, min 427). > > > > On Mon, Mar 28, 2011 at 3:32 PM, Andreas Pauley <apauley@REDACTED> > wrote: > >> > >> Hi Tristan, > >> > >> I tried the code in your repo, but it wasn't significantly faster on my > >> machine. > >> > >> It was interesting though to go through your refactorings. > >> I see you changed the signature for solve so that the empty puzzle and > >> the list of square names are passed in from the start. > >> Maybe I should try this approach in conjunction with the conversion to > >> atoms... > >> > >> Regards, > >> Andreas > >> > >> On Sat, Mar 26, 2011 at 10:56 PM, Tristan Sloughter > >> <tristan.sloughter@REDACTED> wrote: > >> > With a few refactorings ( > https://github.com/tsloughter/sudoku-in-erlang > >> > -- > >> > note I used ewl_plists just to make the code simpler: > >> > https://github.com/erlware/erlware/tree/master/lib/ewlib) and compile > >> > with > >> > native, hipe o3 it beats python's: > >> > $ ./sudoku.py > >> > All tests pass. > >> > Solved 50 of 50 puzzles from easy50.txt in 0.529119 secs (94.50 Hz) > >> > (33059 total eliminations, avg 661.00, median 648, max 830, min > 648). > >> > Solved 95 of 95 puzzles from top95.txt in 3.889944 secs (24.42 Hz) > >> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, min > >> > 648). > >> > Solved 11 of 11 puzzles from hardest.txt in 0.157520 secs (69.83 Hz) > >> > (9436 total eliminations, avg 857.00, median 817, max 1198, min > 648). > >> > $ ./sudoku > >> > All tests passed :-) > >> > Solved 50 of 50 puzzles from easy50.txt in 0.556385 secs (89.87 Hz) > >> > (92538 total eliminations, avg 1850.76, median 1811, max 2628, min > >> > 1767). > >> > Solved 95 of 95 puzzles from top95.txt in 3.593476 secs (26.44 Hz) > >> > (901201 total eliminations, avg 9486.33, median 6267, max 56820, min > >> > 1792). > >> > Solved 11 of 11 puzzles from hardest.txt in 0.162107 secs (67.86 Hz) > >> > (33653 total eliminations, avg 3059.36, median 3023, max 5346, min > >> > 1786). > >> > I ran a number of times, those are pretty much what it always comes > out > >> > to. > >> > But I haven't really tried a full refactoring, I think a number of > >> > things > >> > can be reduced to less iterations. > >> > > >> > Tristan > >> > On Sat, Mar 26, 2011 at 3:39 PM, Ahmed Omar <spawn.think@REDACTED> > >> > wrote: > >> >> > >> >> BTW, beside looking into reducing eleminations (which requires deep > >> >> dive), > >> >> if u do some time profiling using eprof u will find that most of the > >> >> time > >> >> spent actually in the cross functions > >> >> sudoku:'-cross/2-lc$^1/1-1-'/4 96558660 37.46 > >> >> 35024918 [ 0.36] > >> >> > >> >> On Sat, Mar 26, 2011 at 9:33 PM, Ahmed Omar <spawn.think@REDACTED> > >> >> wrote: > >> >>> > >> >>> actually instead of doing > >> >>> > >> >>> NonUniquePeers = shallow_flatten([S || S <- units(Square)]), > >> >>> > >> >>> > >> >>> PeerSet = sets:from_list(NonUniquePeers), > >> >>> > >> >>> > >> >>> PeersWithSelf = sets:to_list(PeerSet), > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> can't u just do > >> >>> PeersWithSelf = lists:usort(NonUniquePeers). > >> >>> > >> >>> > >> >>> ? > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Sat, Mar 26, 2011 at 9:14 PM, Evans, Matthew <mevans@REDACTED > > > >> >>> wrote: > >> >>>> > >> >>>> Good call....gb_sets should be faster. Compiled to native it runs > >> >>>> faster > >> >>>> still > >> >>>> > >> >>>> 9> sudoku:print_results("top95.txt", "\n"). > >> >>>> Solved 95 of 95 puzzles from top95.txt in 0.810156 secs (117.26 Hz) > >> >>>> (901201 total eliminations, avg 9486.33, median 6267, max 56820, > min > >> >>>> 1792). > >> >>>> ok > >> >>>> > >> >>>> ________________________________________ > >> >>>> From: Ahmed Omar [spawn.think@REDACTED] > >> >>>> Sent: Saturday, March 26, 2011 3:08 PM > >> >>>> To: Andreas Pauley > >> >>>> Cc: Evans, Matthew; erlang-questions@REDACTED > >> >>>> Subject: Re: [erlang-questions 20] Re: A sudoku solver in Erlang > >> >>>> compared to Python > >> >>>> > >> >>>> using gb_sets instead of sets could decrease eliminations a bit and > >> >>>> give > >> >>>> u some boost. However, i didn't dive deeper into the code or the > >> >>>> algorithm. > >> >>>> > >> >>>> On Sat, Mar 26, 2011 at 9:41 AM, Andreas Pauley > >> >>>> <apauley@REDACTED<mailto:apauley@REDACTED>> wrote: > >> >>>> Thanks, I've changed the compile options and this definitely makes > it > >> >>>> somewhat faster: > >> >>>> > >> >>>> > >> >>>> > https://github.com/apauley/sudoku-in-erlang/commit/b7ce2abb6ca013850ed8f3e8fd7f5f6be7004cbb > >> >>>> > >> >>>> Strangely the native flag is not documented here: > >> >>>> http://www.erlang.org/doc/man/compile.html#file-2 > >> >>>> > >> >>>> The more interesting improvement would be to decrease the number of > >> >>>> eliminations performed, but for that I'll have to go deep into the > >> >>>> code :-) > >> >>>> > >> >>>> > >> >>>> On Sat, Mar 26, 2011 at 12:16 AM, Evans, Matthew > >> >>>> <mevans@REDACTED<mailto:mevans@REDACTED>> wrote: > >> >>>> > Without going deep into the code, one thing to try is compile > with > >> >>>> > the > >> >>>> > native flag (running the tests from the VM shell here): > >> >>>> > > >> >>>> > Without native set: > >> >>>> > 18> c(sudoku). > >> >>>> > {ok,sudoku} > >> >>>> > 19> sudoku:print_results("top95.txt", "\n"). > >> >>>> > Solved 95 of 95 puzzles from top95.txt in 2.038825 secs (46.60 > Hz) > >> >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, > >> >>>> > min > >> >>>> > 1797). > >> >>>> > ok > >> >>>> > > >> >>>> > > >> >>>> > With native set: > >> >>>> > 20> c(sudoku,[native]). > >> >>>> > {ok,sudoku} > >> >>>> > 21> sudoku:print_results("top95.txt", "\n"). > >> >>>> > Solved 95 of 95 puzzles from top95.txt in 1.613416 secs (58.88 > Hz) > >> >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, > >> >>>> > min > >> >>>> > 1797). > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > -----Original Message----- > >> >>>> > From: > >> >>>> > > >> >>>> > erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED> > >> >>>> > > >> >>>> > [mailto:erlang-questions-bounces@REDACTED<mailto: > erlang-questions-bounces@REDACTED>] > >> >>>> > On Behalf Of Andreas Pauley > >> >>>> > Sent: Friday, March 25, 2011 8:15 AM > >> >>>> > To: erlang-questions@REDACTED<mailto: > erlang-questions@REDACTED> > >> >>>> > Subject: [erlang-questions 4] A sudoku solver in Erlang compared > to > >> >>>> > Python > >> >>>> > > >> >>>> > Hi all, > >> >>>> > > >> >>>> > In my quest to learn Erlang I've translated Peter Norvig's sudoku > >> >>>> > solver into Erlang: > >> >>>> > https://github.com/apauley/sudoku-in-erlang > >> >>>> > > >> >>>> > For comparison, my slightly modified version of Norvig's Python > >> >>>> > code > >> >>>> > can be found here: > >> >>>> > https://github.com/apauley/sudoku-by-norvig > >> >>>> > > >> >>>> > I like the pattern matching, it was fun to do the entire solution > >> >>>> > without a single if or case statement :-) > >> >>>> > Something that bothers me though is that the Python solution > seems > >> >>>> > to > >> >>>> > be faster, even after I've made the Erlang solution use multiple > >> >>>> > processors. > >> >>>> > > >> >>>> > In order to compare the two solutions I counted the number of > >> >>>> > eliminations each performed. > >> >>>> > The eliminate function is the core of the solution, for example > >> >>>> > assigning a single value to a square is implemented as the > >> >>>> > elimination > >> >>>> > of all other values. > >> >>>> > > >> >>>> > With the Erlang implementation I get: > >> >>>> > sudoku-in-erlang$ ./sudoku > >> >>>> > All tests passed :-) > >> >>>> > Solved 50 of 50 puzzles from easy50.txt in 2.890978 secs (17.30 > Hz) > >> >>>> > (93403 total eliminations, avg 1868.06, median 1810, max 2517, > min > >> >>>> > 1770). > >> >>>> > Solved 95 of 95 puzzles from top95.txt in 22.004369 secs (4.32 > Hz) > >> >>>> > (922678 total eliminations, avg 9712.40, median 6596, max 55370, > >> >>>> > min > >> >>>> > 1797). > >> >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.851678 secs (12.92 > >> >>>> > Hz) > >> >>>> > (32339 total eliminations, avg 2939.91, median 2894, max 4779, > min > >> >>>> > 1781). > >> >>>> > > >> >>>> > And with the Python implementation: > >> >>>> > sudoku-by-norvig$ ./sudoku.py > >> >>>> > All tests pass. > >> >>>> > Solved 50 of 50 puzzles from easy50.txt in 0.792008 secs (63.13 > Hz) > >> >>>> > (33059 total eliminations, avg 661.00, median 648, max 830, min > >> >>>> > 648). > >> >>>> > Solved 95 of 95 puzzles from top95.txt in 5.903875 secs (16.09 > Hz) > >> >>>> > (221997 total eliminations, avg 2336.00, median 1492, max 11512, > >> >>>> > min > >> >>>> > 648). > >> >>>> > Solved 11 of 11 puzzles from hardest.txt in 0.237532 secs (46.31 > >> >>>> > Hz) > >> >>>> > (9436 total eliminations, avg 857.00, median 817, max 1198, min > >> >>>> > 648). > >> >>>> > > >> >>>> > So according to the stats above, the Python solution performs > less > >> >>>> > computations when given exactly the same input. > >> >>>> > The Erlang code is as close to the Python as I could make it, > I've > >> >>>> > done more or less a direct translation of the algorithms used. > >> >>>> > > >> >>>> > I suspect that there are some lazy evaluation happening in the > >> >>>> > Python > >> >>>> > version, possibly generators, although I haven't pinpointed it > yet. > >> >>>> > > >> >>>> > How can I improve my Erlang code in this solution? > >> >>>> > > >> >>>> > Kind regards, > >> >>>> > Andreas > >> >>>> > _______________________________________________ > >> >>>> > erlang-questions mailing list > >> >>>> > erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >> >>>> > http://erlang.org/mailman/listinfo/erlang-questions > >> >>>> > > >> >>>> _______________________________________________ > >> >>>> erlang-questions mailing list > >> >>>> erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> > >> >>>> http://erlang.org/mailman/listinfo/erlang-questions > >> >>>> > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> Best Regards, > >> >>>> - Ahmed Omar > >> >>>> http://nl.linkedin.com/in/adiaa > >> >>>> Follow me on twitter > >> >>>> @spawn_think<http://twitter.com/#!/spawn_think> > >> >>>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Best Regards, > >> >>> - Ahmed Omar > >> >>> http://nl.linkedin.com/in/adiaa > >> >>> Follow me on twitter > >> >>> @spawn_think > >> >> > >> >> > >> >> > >> >> -- > >> >> Best Regards, > >> >> - Ahmed Omar > >> >> http://nl.linkedin.com/in/adiaa > >> >> Follow me on twitter > >> >> @spawn_think > >> >> > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> >> > >> > > >> > > > > > > > > > -- > > Best Regards, > > - Ahmed Omar > > http://nl.linkedin.com/in/adiaa > > Follow me on twitter > > @spawn_think > > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/efdc9e9d/attachment.htm> From t.dt.aanderson@REDACTED Tue Mar 29 13:08:57 2011 From: t.dt.aanderson@REDACTED (Thomas Anderson) Date: Tue, 29 Mar 2011 19:08:57 +0800 Subject: [erlang-questions] Re: Internal mechanism of monitor In-Reply-To: <AANLkTim7Yz+1NO6tRdQH8hbrp4E2+nre4a8Qwpqu29kQ@mail.gmail.com> References: <AANLkTimBnPPmFqdFjmZ+cuNtbkF1k455yxE+BmC6qgve@mail.gmail.com> <AANLkTim7Yz+1NO6tRdQH8hbrp4E2+nre4a8Qwpqu29kQ@mail.gmail.com> Message-ID: <AANLkTim2MbG20xhVK=O7s+3jiODP4kzwvw+Z83KZmimY@mail.gmail.com> Thank you for the concise explain. It is very clear and useful! One more question. I read that supervisor behavior is applied for start/ stop child process with different strategy (one for one, one for all, etc.) In addition to such strategy is there any other behavior (pattern) that is usually applied regarding to the fault tolerance? Thanks again for kindly help. I appreciate it. On Tue, Mar 29, 2011 at 5:08 PM, Attila Rajmund Nohl <attila.r.nohl@REDACTED> wrote: > 2011/3/29, Thomas Anderson <t.dt.aanderson@REDACTED>: >> I read some articles talking about erlang and am caught attention by >> erlang's intrinsically fault tolerance features. >> >> In the paper `Making reliable distributed system in the presence of >> software error,' it explains error handling with e.g links and >> monitor. But it only refers to function such as link(), spwan_link(), >> monitor(), etc., and some basic illustration about these two >> functions. After searching on erlang.org as well as the internet I do >> not find document or threads discussing its internal mechanism. For >> instance, is it failure detector? > > The documentation says that > > "erlang:monitor(Type, Item) -> MonitorRef > ... > The calling process starts monitoring Item which is an object of type > Type. Currently only processes can be monitored, i.e. the only allowed > Type is process > ... > A ?'DOWN' message will be sent to the monitoring process if Item dies, > if Item does not exist, or if the connection is lost to the node which > Item resides on." > > So it's a failure detector. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From krab@REDACTED Tue Mar 29 13:40:35 2011 From: krab@REDACTED (Kresten Krab Thorup) Date: Tue, 29 Mar 2011 13:40:35 +0200 Subject: [erlang-questions] Re: [erlang-questions 41] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <AANLkTikQr8-OLPWUUn9Fi=xP9OTJhyeqYUABXhC=Hr=x@mail.gmail.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> <AANLkTikQr8-OLPWUUn9Fi=xP9OTJhyeqYUABXhC=Hr=x@mail.gmail.com> Message-ID: <36321687-C8A5-4E70-A3A8-3D9CBB90458A@trifork.com> Hi, On Mar 28, 2011, at 15:22 , Andreas Pauley wrote: > I'd love to use atoms as square names, just because it looks cleaner. > However, I'm getting a severe performance hit when I use > list_to_atom/1 on each square name. > > Can you show me how you did it in conjunction with ct_expand? These are the changed functions, [and then the unit tests need to be updated accordingly]... ct_expand doesn't like function applications, so the function unitlist() needs to have the other definitions inlined. Kresten squares() -> %% Returns a list of 81 square names, including "a1" etc. ct_expand:term([erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- ?cols]). col_squares() -> %% All the square names for each column. ([[erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- [C]] || C <- ?cols]). row_squares() -> %% All the square names for each row. ct_expand:term([[erlang:list_to_atom([X,Y]) || X <- [R], Y <- ?cols] || R <- ?rows]). box_squares() -> %% All the square names for each box. ct_expand:term([[erlang:list_to_atom([X,Y]) || X <- R, Y <- C] || R <- ["abc", "def", "ghi"], C <- ["123", "456", "789"]]). unitlist() -> %% A list of all units (columns, rows, boxes) in a grid. ct_expand:term( [[erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- [C]] || C <- ?cols] ++ [[erlang:list_to_atom([X,Y]) || X <- [R], Y <- ?cols] || R <- ?rows] ++ [[erlang:list_to_atom([X,Y]) || X <- R, Y <- C] || R <- ["abc", "def", "ghi"], C <- ["123", "456", "789"]]). From raimo+erlang-questions@REDACTED Tue Mar 29 14:41:53 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 29 Mar 2011 14:41:53 +0200 Subject: [erlang-questions] Re: [erlang-questions 90] Re: New Mailing list software In-Reply-To: <80F7351E-F36A-4B99-B5BB-1EA0C871616E@sol42.com> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <4D8D0A5E.3000208@telegraphics.com.au> <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> <20110328073551.GB13069@erix.ericsson.se> <20110328075701.GA6079@mulga.csse.unimelb.edu.au> <20110328144922.GA26511@erix.ericsson.se> <80F7351E-F36A-4B99-B5BB-1EA0C871616E@sol42.com> Message-ID: <20110329124153.GA28450@erix.ericsson.se> On Mon, Mar 28, 2011 at 09:58:13PM +0200, DS wrote: > Hello. While we are at it, one more thing :) > Not really a mailing list software issue: the machine that sends this list's messages claims to be "hades.cslab.ericsson.net", but according to the DNS the IP address is that of "ns.cslab.ericsson.net". This mismatch is a very common trait of zombies and spamming machines. My SMTP server takes RFC 2821 very, maybe even too seriously, and barks every time a message is received, and that is after telling it to actually accept the connection: before that it would simply drop it (violating said RFC 2821, but it works wonders against spam) and bark as well. Does this have an easy fix? > Regards. > -Daniel Hints are welcome. The machine name is configured as hades.cslab.ericsson.net and the IP address in DNS is 192.121.151.104. It also has a few other IP addresses e.g ns.cslab.ericsson.se: 192.121.151.103. Postfix is configured to use hades.cslab.ericsson.net. But it seems outgoing sockets nevertheless appears to come from 192.121.151.103. I will try to change the IP address order in /etc/rc.conf to see if it can affect which address is perceived to be the real address... -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Tue Mar 29 14:44:50 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 29 Mar 2011 14:44:50 +0200 Subject: [erlang-questions] Re: [erlang-questions 11] fun with the new erlang.org! In-Reply-To: <20110325185008.GA32567@ecn.lan> References: <20110325185008.GA32567@ecn.lan> Message-ID: <20110329124450.GB28450@erix.ericsson.se> On Fri, Mar 25, 2011 at 02:50:08PM -0400, Michael Santos wrote: > > http://www.erlang.org/<PLAINTEXT> > > And the WINNAR! > > http://www.erlang.org/</div><div><h1 style="font-size:60px;">Powered by node.js</h1></div> > > ;) Ok. Now that should be fixed. Thank you for pointing this! Please report if anyone finds anything else... > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From spawn.think@REDACTED Tue Mar 29 14:59:52 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Tue, 29 Mar 2011 14:59:52 +0200 Subject: [erlang-questions] Re: [erlang-questions 8] Re: code_server:call/2 problem? In-Reply-To: <OFD4B617D5.EAEAE42C-ONC125785E.005EB9EC-C125785E.005F7DC8@notes.edfgdf.fr> References: <AANLkTi=DmBcvym3WB6O07+5XrTj=UGunQs3tN=XJMvqA@mail.gmail.com> <OFD4B617D5.EAEAE42C-ONC125785E.005EB9EC-C125785E.005F7DC8@notes.edfgdf.fr> Message-ID: <AANLkTim=YM8psyRc6sGWE0i-vQgX3ro40sC4HOSf1njO@mail.gmail.com> Any explanation about the supervisor children order in kernel here? why rpc first? On Fri, Mar 25, 2011 at 6:23 PM, Olivier BOUDEVILLE < olivier.boudeville@REDACTED> wrote: > > Hi, > > By the way, looking at the article about Dialyzer's race condition > detection capabilities ( > http://www.it.uu.se/research/group/hipe/dialyzer/publications/races.pdf), > I was surprised that what appears like a race condition involving > code_server was not spotted, whereas it seemed to correspond to the process > registry-based race condition type described in the article (first > documented example). > > Unless it corresponds to one of the 4 "ProcR" conditions established for > kernel, and it has not been fixed yet? > > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > *spawn.think@REDACTED* > Envoy? par : erlang-questions@REDACTED > > 21/03/2011 14:46 > A > olivier.boudeville@REDACTED > cc > mevans@REDACTED, erlang-questions@REDACTED > Objet > Re: [erlang-questions] code_server:call/2 problem? > > > > > Hi Olivier, > I think it has to do with the supervisor children order in kernel app. > currently the rpc server is started before the code_server, i'm not sure if > there's a good reason for that > > {ok, {SupFlags, > [Rpc, Global, InetDb | DistAC] ++ > [NetSup, Glo_grp, File, Code, > StdError, User, Config, SafeSupervisor] ++ Timer}} > > > On Mon, Mar 21, 2011 at 1:22 PM, Olivier BOUDEVILLE < > olivier.boudeville@REDACTED> wrote: > > > Hi, > > > > Thanks for your answer. Indeed, this could have been explanation; however > > the node is crashing after that error, not before nor "in parallel" to > the > > error. > > > > Actually I believe there is a bug in the Erlang runtime. I strongly > > suspect there is a small time window during which a race condition can > > occur: apparently code:load_binary can be triggered (thanks to > > rpc:multicall) on a just-launched node before at least one of its system > > processes succeeds in registering its name. At least that's what I came > to > > think after having peered at lib/kernel/src/code_server.erl, the badarg > > that occured may come from the fact that call/2 is called whereas Name is > > not registered (yet), in: > > > > """ > > call(Name, Req) -> > > Name ! {code_call, self(), Req}, > > receive > > {?MODULE, Reply} -> > > Reply > > end. > > """" > > > > As a test, the non-systematic crash which, on our short test case (done > on > > Ubuntu 64-bit running on a 4--core Core i7 laptop), was taking on average > > 30 seconds (loop of ~15 attempts) before happening, never happened with > > the same loop being run for more than one hour, once I inserted a > > timer:sleep(1000) in my deployment manager between the launching of the > > remote VM and the call to rpc:multicall (knowing that intermediate > > checkings like Erlang ping of the remote node and checking of the remote > > Erlang version always succeeded). > > > > I suppose there is in the runtime a kind of synchronous barrier where all > > system processes are checked to be up and ready (including appropriately > > registered) before serving user-space requests, but probably that at > least > > one system process was forgotten and thus led to such a race condition. > > Unless I am mistaken? > > > > Thanks in advance for any answer, > > Best regards, > > > > Olivier. > > --------------------------- > > Olivier Boudeville > > > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > > 65 27 13 > > > > > > > > mevans@REDACTED > > Envoy? par : erlang-questions@REDACTED > > 18/03/2011 19:22 > > > > A > > olivier.boudeville@REDACTED, erlang-questions@REDACTED > > cc > > > > Objet > > RE: [erlang-questions] code_server:call/2 problem? > > > > > > > > > > > > > > Do you see any crashes on the remote nodes? > > > > It does look like the remote code_server application has got a request, > > but it for some reason fails (corrupted data perhaps?). I'm wondering if > > you could attach a remote shell to one of those nodes and trace the > > code_server module? > > > > -----Original Message----- > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] > On > > Behalf Of Olivier BOUDEVILLE > > Sent: Friday, March 18, 2011 1:32 PM > > To: erlang-questions@REDACTED > > Subject: [erlang-questions] code_server:call/2 problem? > > > > Hi, > > > > We are running a distributed Erlang program on a user node from which a > > number of computing nodes are spawned, via SSH for the remote hosts. To > > perform the automatic deployment, two deployment-related modules are sent > > to each of the spawned nodes, using the traditional approach (first a > call > > > > to code:get_object_code/1 then a rpc:multicall of code:load_binary). > > > > However, sometimes (not frequently), with the exact same settings, the > > first module cannot be deployed successfully. We have indeed: > > > > {ResList,BadNodes} = rpc:multicall( Nodes, code, load_binary, [ > > ModuleName, ModuleFilename, ModuleBinary ], Timeout ), > > > > that returns: > > ResList = > > > > > [{badrpc,{'EXIT',{badarg,[{code_server,call,2},{rpc,'-handle_call_call/6-fun-0-',5}]}}}] > > BadNodes = [] > > > > This happens with R14B02, but most probably with previous versions as > > well. > > Apparently this happens often (always?) on a node created on the user > > host. > > I am pretty sure the deployed node is "fresh" (blank, vanilla). > > And ignoring the badrpc will result in a undef error as soon as the first > > function of the first helper module is called, even if delaying the call > > (a race condition was suspected if ever the actual loading was > > asynchronous). > > > > Would anyone see a cause for such a badarg non-systematic error? > > > > Thanks in advance for any hint, > > Best regards, > > > > Olivier Boudeville. > > --------------------------- > > Olivier Boudeville > > > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > > 65 27 13 > > > > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > > ?tablis ? l'intention exclusive des destinataires et les informations qui > > y figurent sont strictement confidentielles. Toute utilisation de ce > > Message non conforme ? sa destination, toute diffusion ou toute > > publication totale ou partielle, est interdite sauf autorisation > expresse. > > > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune > trace > > sur quelque support que ce soit. Nous vous remercions ?galement d'en > > avertir imm?diatement l'exp?diteur par retour du message. > > > > Il est impossible de garantir que les communications par messagerie > > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > > erreur ou virus. > > ____________________________________________________ > > > > This message and any attachments (the 'Message') are intended solely for > > the addressees. The information contained in this Message is > confidential. > > Any use of information contained in this Message not in accord with its > > purpose, any dissemination or disclosure, either whole or partial, is > > prohibited except formal approval. > > > > If you are not the addressee, you may not copy, forward, disclose or use > > any part of it. If you have received this message in error, please delete > > it and all copies from your system and notify the sender immediately by > > return message. > > > > E-mail communication cannot be guaranteed to be timely secure, error or > > virus-free. > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > > ?tablis ? l'intention exclusive des destinataires et les informations qui > y > > figurent sont strictement confidentielles. Toute utilisation de ce > Message > > non conforme ? sa destination, toute diffusion ou toute publication > totale > > ou partielle, est interdite sauf autorisation expresse. > > > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune > trace > > sur quelque support que ce soit. Nous vous remercions ?galement d'en > avertir > > imm?diatement l'exp?diteur par retour du message. > > > > Il est impossible de garantir que les communications par messagerie > > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > > erreur ou virus. > > ____________________________________________________ > > > > This message and any attachments (the 'Message') are intended solely for > > the addressees. The information contained in this Message is > confidential. > > Any use of information contained in this Message not in accord with its > > purpose, any dissemination or disclosure, either whole or partial, is > > prohibited except formal approval. > > > > If you are not the addressee, you may not copy, forward, disclose or use > > any part of it. If you have received this message in error, please delete > it > > and all copies from your system and notify the sender immediately by > return > > message. > > > > E-mail communication cannot be guaranteed to be timely secure, error or > > virus-free. > > > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think <http://twitter.com/#!/spawn_think> > > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui y > figurent sont strictement confidentielles. Toute utilisation de ce Message > non conforme ? sa destination, toute diffusion ou toute publication totale > ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete it > and all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think <http://twitter.com/#!/spawn_think> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/3d6fc1db/attachment.htm> From apauley@REDACTED Tue Mar 29 15:13:59 2011 From: apauley@REDACTED (Andreas Pauley) Date: Tue, 29 Mar 2011 15:13:59 +0200 Subject: [erlang-questions] Re: [erlang-questions 41] Re: A sudoku solver in Erlang compared to Python In-Reply-To: <36321687-C8A5-4E70-A3A8-3D9CBB90458A@trifork.com> References: <AANLkTik1fW4RBYuy-RCaZfUp1u2rJZnJRF0t0bJQ3VcU@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAB2FC214@vvexch.verivue.com> <AANLkTi=FDxkbv4AJ38P1jePUOenco3q=iDhp_Qkb84Gj@mail.gmail.com> <AANLkTik2MuVZVUuw7T+kTC4pxq3T6FWVWFDmhQ9EyXS8@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB28@vvexch.verivue.com> <AANLkTi=jRvK31jxgX-_Yz+OQA_SEfKzfBz+zxazsRnnE@mail.gmail.com> <AANLkTikZx-F67F=dtZr-sio4BJrO8XTfd2zpzAeVqZge@mail.gmail.com> <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB2A@vvexch.verivue.com> <DD456FDB-6D5C-4A72-977A-606F8A39F6F3@trifork.com> <AANLkTikQr8-OLPWUUn9Fi=xP9OTJhyeqYUABXhC=Hr=x@mail.gmail.com> <36321687-C8A5-4E70-A3A8-3D9CBB90458A@trifork.com> Message-ID: <AANLkTi=mTAP_05EbubT4WFercbYSwoWx8qzH7b5hc4oB@mail.gmail.com> Thanks! https://github.com/apauley/sudoku-in-erlang/commit/9c69a9d813a9c5e0b6e41471eb3e3df5ea284a4d The 3 inlined functions are now dead, so I just removed them. On Tue, Mar 29, 2011 at 1:40 PM, Kresten Krab Thorup <krab@REDACTED> wrote: > Hi, > > On Mar 28, 2011, at 15:22 , Andreas Pauley wrote: > >> I'd love to use atoms as square names, just because it looks cleaner. >> However, I'm getting a severe performance hit when I use >> list_to_atom/1 on each square name. >> >> Can you show me how you did it in conjunction with ct_expand? > > These are the changed functions, [and then the unit tests need to be updated accordingly]... ?ct_expand doesn't like function applications, so the function unitlist() needs to have the other definitions inlined. > > Kresten > > > squares() -> > ? ?%% Returns a list of 81 square names, including "a1" etc. > ? ?ct_expand:term([erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- ?cols]). > col_squares() -> > ? ?%% All the square names for each column. > ? ?([[erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- [C]] || C <- ?cols]). > row_squares() -> > ? ?%% All the square names for each row. > ? ?ct_expand:term([[erlang:list_to_atom([X,Y]) || X <- [R], Y <- ?cols] || R <- ?rows]). > box_squares() -> > ? ?%% All the square names for each box. > ? ?ct_expand:term([[erlang:list_to_atom([X,Y]) || X <- R, Y <- C] || > ? ? ? ? ? ? ? ? ? ? ? R <- ["abc", "def", "ghi"], > ? ? ? ? ? ? ? ? ? ? ? C <- ["123", "456", "789"]]). > > unitlist() -> > ? ?%% A list of all units (columns, rows, boxes) in a grid. > ? ?ct_expand:term( ?[[erlang:list_to_atom([X,Y]) || X <- ?rows, Y <- [C]] || C <- ?cols] > ? ? ? ? ? ? ? ? ? ? ++ [[erlang:list_to_atom([X,Y]) || X <- [R], Y <- ?cols] || R <- ?rows] > ? ? ? ? ? ? ? ? ? ? ++ [[erlang:list_to_atom([X,Y]) || X <- R, Y <- C] || > ? ? ? ? ? ? ? ? ? ? ? ? ? ?R <- ["abc", "def", "ghi"], > ? ? ? ? ? ? ? ? ? ? ? ? ? ?C <- ["123", "456", "789"]]). > > From raimo+erlang-questions@REDACTED Tue Mar 29 15:22:13 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 29 Mar 2011 15:22:13 +0200 Subject: [erlang-questions] Re: [erlang-questions 90] Re: New Mailing list software In-Reply-To: <20110329124153.GA28450@erix.ericsson.se> References: <9688EB4F-9F34-4486-9E3B-FE60A47B99B1@gmail.com> <4D8D0A5E.3000208@telegraphics.com.au> <D51E4C8B-8FAA-4B0C-B986-C3E1C9017568@kent.ac.uk> <20110328073551.GB13069@erix.ericsson.se> <20110328075701.GA6079@mulga.csse.unimelb.edu.au> <20110328144922.GA26511@erix.ericsson.se> <80F7351E-F36A-4B99-B5BB-1EA0C871616E@sol42.com> <20110329124153.GA28450@erix.ericsson.se> Message-ID: <20110329132213.GA29716@erix.ericsson.se> On Tue, Mar 29, 2011 at 02:41:53PM +0200, Raimo Niskanen wrote: > On Mon, Mar 28, 2011 at 09:58:13PM +0200, DS wrote: > > Hello. While we are at it, one more thing :) > > Not really a mailing list software issue: the machine that sends this list's messages claims to be "hades.cslab.ericsson.net", but according to the DNS the IP address is that of "ns.cslab.ericsson.net". This mismatch is a very common trait of zombies and spamming machines. My SMTP server takes RFC 2821 very, maybe even too seriously, and barks every time a message is received, and that is after telling it to actually accept the connection: before that it would simply drop it (violating said RFC 2821, but it works wonders against spam) and bark as well. Does this have an easy fix? > > Regards. > > -Daniel > > Hints are welcome. The machine name is configured as hades.cslab.ericsson.net > and the IP address in DNS is 192.121.151.104. It also has a few other IP > addresses e.g ns.cslab.ericsson.se: 192.121.151.103. Postfix is configured > to use hades.cslab.ericsson.net. > > But it seems outgoing sockets nevertheless appears to come from > 192.121.151.103. > > I will try to change the IP address order in /etc/rc.conf to see if > it can affect which address is perceived to be the real address... A reboot, and I think it worked! At least now netstat tells me that outgoing SMTP sockets originate from 192.121.151.104. But I did not check before the reboot... > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jwest@REDACTED Tue Mar 29 18:57:12 2011 From: jwest@REDACTED (Jordan West) Date: Tue, 29 Mar 2011 09:57:12 -0700 Subject: [erlang-questions] Re: Common Test Question In-Reply-To: <AANLkTimiHfYK8g8AMFo9PgJJfzG2CC90X0Af+ARrV3_d@mail.gmail.com> References: <BANLkTi=g8rsUgv-tZUw1bALiVd5BWPViaQ@mail.gmail.com> <AANLkTimiHfYK8g8AMFo9PgJJfzG2CC90X0Af+ARrV3_d@mail.gmail.com> Message-ID: <BANLkTi==BeCsVvr_py-RX_28i0=CRzZnqg@mail.gmail.com> Peter, Thanks for letting me know. I considered it a bug as well. On the topic of the config functions, I think other testing frameworks like Ruby's RSpec ignore them when no tests in the group exist. I assume this is to prevent hard to debug side-effects in tests and as a small optimization. Jordan West On Tue, Mar 29, 2011 at 1:56 AM, Peter Andersson <pekadan@REDACTED> wrote: > Hi Jordan, > > I learned about this problem just the other day. I consider it a bug > really. An empty group should be ignored, or possibly, the config > functions for the group should be executed (even if there's no test > case to configure for). I have written a ticket on it and a fix will > come shortly. > > Thanks for reporting! > > Best regards, > Peter > > Ericsson AB, Erlang/OTP > > 2011/3/29 Jordan West <jwest@REDACTED>: > > I've recently been using common_test on several new projects and am > really > > liking it. I ran across something today I found odd, however, and wanted > to > > know if it was the expected behavior. I was expecting that if I have a > > group with no tests that group should, essentially, be ignored. This > seems > > to be the case if the group is contained in another but not if the group > is > > an element in the list returned by all/0. In the latter case the suite > > errors out with "Error in suite detected: 'Invalid reference to group > [GROUP > > NAME] in [SUITE NAME]:all/0'". > > Is this intentional and if so, why are sub-groups treated differently? Is > > there a reason its not a warning instead? For the most part I'm just > > curious. I do like setting up my groups sometimes ahead of writing tests > and > > ran into this when I tried to leave a top-level group empty for a short > > while. > > A minimal code example is below. I'm using R14B02 on OS-X with > common_test > > 1.5.3 (I installed today off the Erlang Factory USB stick, thanks for a > > great conference everyone). Oh, and this is my first message to the list. > > Hello, all. > > %% a.erl > > -module(a). > > -compile(export_all). > > my_fun() -> > > 1. > > %% a_SUITE.erl > > -module(a_SUITE). > > -compile(export_all). > > groups() -> > > [{group_1, > > [shuffle], > > [test_1, {group, group_3}]}, > > {group_2, > > [shuffle], > > []}, % this empty test list causes issues if empty > > {group_3, > > [],[]}]. % this empty test list seems to be able to be empty > > all() -> > > [ > > {group, group_1}, > > {group, group_2} %% If this is remove the suite won't error out > > ]. > > test_1(_) -> > > 1 = a:my_fun(). > >>> run_test -suite a_SUITE.erl > > ... > > Error in suite detected: 'Invalid reference to group group_2 in > > a_SUITE:all/0' > > Jordan West > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/9234ff16/attachment.htm> From rtrlists@REDACTED Tue Mar 29 19:00:34 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 29 Mar 2011 18:00:34 +0100 Subject: [erlang-questions] Re: Differences in Erlang shells in Windows In-Reply-To: <45C0EEFF-934E-4555-AECB-69B5D2D69C23@gmail.com> References: <20110328145403.47OWY.216474.imail@eastrmwml32> <45C0EEFF-934E-4555-AECB-69B5D2D69C23@gmail.com> Message-ID: <AANLkTimPEsSB_eTTbqo=TOc3MtO-UQnSPn5Z3TXSP4bw@mail.gmail.com> On Mon, Mar 28, 2011 at 8:22 PM, Koener Antoine <antoine.koener@REDACTED>wrote: > is of the shells "erl.exe" and "werl.exe". I'm kind of new to this, and I >> want to make sure I am using the right one. >> > > Easy: > erl.exe don't handle completion, (i.e. TAB is a \t) and runs in 'cmd.exe' > werl.exe is autonomous, you can change the font, you have completion (TAB > completes) > > erl.exe is needed to launch a service (see erlsrv) > werl.exe is the prefered window shell. > > Finally experiment yourself, you'll find that werl is the only way to go on > windows... > > And erl.exe doesn't handle Ctrl-G, whereas werl.exe does. Ctrl-G brings you into the "Shell Break Mode", where you can stop, start, and switch shells. "Erlang and OTP in Action" by Logan, Merrit, and Carlsson, Chapter 2, has a decent description of what you can do in that mode. Pretty neat, especially when doing distributed stuff. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/e45818b2/attachment.htm> From bsvancara@REDACTED Tue Mar 29 19:00:45 2011 From: bsvancara@REDACTED (Bohuslav Svancara) Date: Tue, 29 Mar 2011 19:00:45 +0200 Subject: [erlang-questions] rb help error Message-ID: <AANLkTikJo0y3ot+QYvYGSbrkg2EqEu+F0YyZ9tHz=JAf@mail.gmail.com> Wrong help in rb: rb:filter(Filters, Dates) - same as rb:filter/1 but accepts date ranges to filter reports. Dates must be of the following form: - {StartDate, EndDate} StartDate = EndDate = {{Y-M-D},{H,M,S}} Should be: StartDate = EndDate = {{Y,M,D},{H,M,S}} Bob -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/2cb0e434/attachment.htm> From rtrlists@REDACTED Tue Mar 29 19:06:16 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 29 Mar 2011 18:06:16 +0100 Subject: [erlang-questions] Re: crypto on 64bit windows? In-Reply-To: <4D91B095.3040600@rabbitmq.com> References: <4D91B095.3040600@rabbitmq.com> Message-ID: <AANLkTikNvGDBaZtjObaUV_jiy2Nbey5Y-fJOyK4KbEYB@mail.gmail.com> On Tue, Mar 29, 2011 at 11:12 AM, Emile Joubert <emile@REDACTED> wrote: > > Hi, > > Should the crypto module work on 64bit windows? It works fine on 32bit > Windows XP, but on 64bit XP and 64bit Windows7 I get the following error: > > ---------- > > Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] > > Eshell V5.8.3 (abort with ^G) > 1> crypto:start(). > > =ERROR REPORT==== 29-Mar-2011::11:05:13 === > Unable to load crypto library. Failed with error: > "load_failed, Failed to load NIF library > c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The specified > module could not be found.'" > OpenSSL might not be installed on this system. > > =ERROR REPORT==== 29-Mar-2011::11:05:13 === > The on_load function for module crypto returned {error, > {load_failed, > "Failed to load NIF > library c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The > specified module could not be found.'"}} > ** exception error: undefined function crypto:start/0 > > ---------- > > C:\Program Files (x86)\erl5.8.3\lib\crypto-2.0.2.1\priv\lib > does contain crypto.dll. Installing a 64bit version of OpenSSL for Windows > doesn't help. > > The crypto documentation refers to Win32 - should I infer that crypto will > not work on 64bit Windows operating systems? > > > > Regards > > Emile > > The crypto.dll needs to find libeay32.dll (on 32-bit systems at least, not sure what it is called on 64-bit) and if it finds the wrong one (an old one for example) then crypto.dll won't load. We've had a lot of fun recently with machines where some rather annoying miscreant application plonked it's version of libeay32.dll in the \Windows\system32 folder. And due to the wonders of Windows DLL loading paths, that takes precedence before anything in your PATH. One way of "solving" this kind of thing is to copy the OpenSSL dlls into the same folder where your erl.exe is getting started from. Cause that in turn takes precedence over the Windows one. Grrr. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/a373b9ea/attachment.htm> From rtrlists@REDACTED Tue Mar 29 19:10:36 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 29 Mar 2011 18:10:36 +0100 Subject: [erlang-questions] Re: crypto on 64bit windows? In-Reply-To: <AANLkTikNvGDBaZtjObaUV_jiy2Nbey5Y-fJOyK4KbEYB@mail.gmail.com> References: <4D91B095.3040600@rabbitmq.com> <AANLkTikNvGDBaZtjObaUV_jiy2Nbey5Y-fJOyK4KbEYB@mail.gmail.com> Message-ID: <AANLkTi=DcfchKRp0ncQujUNWs0Lh8gBYdQsv3YFFhhWM@mail.gmail.com> On Tue, Mar 29, 2011 at 6:06 PM, Robert Raschke <rtrlists@REDACTED>wrote: > > On Tue, Mar 29, 2011 at 11:12 AM, Emile Joubert <emile@REDACTED>wrote: > >> >> Hi, >> >> Should the crypto module work on 64bit windows? It works fine on 32bit >> Windows XP, but on 64bit XP and 64bit Windows7 I get the following error: >> >> ---------- >> >> Erlang R14B02 (erts-5.8.3) [smp:2:2] [rq:2] [async-threads:0] >> >> Eshell V5.8.3 (abort with ^G) >> 1> crypto:start(). >> >> =ERROR REPORT==== 29-Mar-2011::11:05:13 === >> Unable to load crypto library. Failed with error: >> "load_failed, Failed to load NIF library >> c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The specified >> module could not be found.'" >> OpenSSL might not be installed on this system. >> >> =ERROR REPORT==== 29-Mar-2011::11:05:13 === >> The on_load function for module crypto returned {error, >> {load_failed, >> "Failed to load NIF >> library c:/PROGRA~2/ERL58~1.3/lib/crypto-2.0.2.1/priv/lib/crypto: 'The >> specified module could not be found.'"}} >> ** exception error: undefined function crypto:start/0 >> >> ---------- >> >> C:\Program Files (x86)\erl5.8.3\lib\crypto-2.0.2.1\priv\lib >> does contain crypto.dll. Installing a 64bit version of OpenSSL for Windows >> doesn't help. >> >> The crypto documentation refers to Win32 - should I infer that crypto will >> not work on 64bit Windows operating systems? >> >> >> >> Regards >> >> Emile >> >> > The crypto.dll needs to find libeay32.dll (on 32-bit systems at least, not > sure what it is called on 64-bit) and if it finds the wrong one (an old one > for example) then crypto.dll won't load. > > We've had a lot of fun recently with machines where some rather annoying > miscreant application plonked it's version of libeay32.dll in the > \Windows\system32 folder. And due to the wonders of Windows DLL loading > paths, that takes precedence before anything in your PATH. > > One way of "solving" this kind of thing is to copy the OpenSSL dlls into > the same folder where your erl.exe is getting started from. Cause that in > turn takes precedence over the Windows one. Grrr. > > Robby > > Apologies, the last paragraph was meant to read: One way of "solving" this kind of thing is to copy the OpenSSL dlls into the same folder where the erl.exe (or werl.exe) that you are running lives. Cause that in turn takes precedence over the Windows one. Grrr. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/1695b4a8/attachment.htm> From sam@REDACTED Wed Mar 30 01:13:33 2011 From: sam@REDACTED (Sam Bobroff) Date: Wed, 30 Mar 2011 10:13:33 +1100 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <AANLkTinZw007nBx4zxVhcypSD3p9nV3MKcV=rUKrVCqN@mail.gmail.com> References: <4D902590.8060409@m5net.com> <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> <AANLkTim0H4KwjKVe9mLzAzAhmdMxr7jiueea1uGoiB3k@mail.gmail.com> <AANLkTinZw007nBx4zxVhcypSD3p9nV3MKcV=rUKrVCqN@mail.gmail.com> Message-ID: <AANLkTim1H6KFjn=8NGC=EispZ=qeBxQ2RJkZfWSEoovN@mail.gmail.com> 2011/3/29 Per Melin <per.melin@REDACTED> > > While we're on the subject, here are a couple of minor gotchas that > could possibly be improved in future versions of reltool: > > 1. Where the .rel is placed. > A release named "foo" with version "001" created with > reltool:create_target/2 in target directory "foo_001" will get this > .rel file: > foo_001/releases/001/foo.rel > > But that is not what release_handler expects. So you need to move and > possibly rename the file to: > foo_001/foo_001.rel > Ah, thanks for that :-) 2. start.boot. > For the same release you get a boot file called foo.boot. But > release_handler will barf unless there is a start.boot (which I > believe is really irrelevant unless you run in embedded mode with the > default scripts). So maybe when you include the option {boot_rel, > "foo"} that could mean that foo.boot will instead be called > start.boot? > Yes. I've successfully used boot_rel to specify my own boot script. It works fine and the specified script is the one that runs if you don't provide one on the command line. Peace, Sam. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/1079b17c/attachment.htm> From jacob.vorreuter@REDACTED Wed Mar 30 01:13:31 2011 From: jacob.vorreuter@REDACTED (Jacob Vorreuter) Date: Tue, 29 Mar 2011 16:13:31 -0700 Subject: [erlang-questions] Erlang position at Heroku Message-ID: <9F2BB5A7-F1C1-4A78-BD90-3FA5C66DDD1E@gmail.com> We are looking for an additional Erlang developer at Heroku: http://jobs.heroku.com/erlang_infrastructure_developer If you're interested, just follow the directions in the post. Feel free to email me with questions. Happy Erlangs! Jake -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/7730757d/attachment.htm> From ok@REDACTED Wed Mar 30 02:38:02 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 30 Mar 2011 13:38:02 +1300 Subject: [erlang-questions] Re: Newbie Erlang programmer looking for feedback! :) In-Reply-To: <c4415989-af65-4fb5-bde1-ff93738e9155@f2g2000yqf.googlegroups.com> References: <c4415989-af65-4fb5-bde1-ff93738e9155@f2g2000yqf.googlegroups.com> Message-ID: <18541270-1F64-434E-9741-E992DCF4C414@cs.otago.ac.nz> Let's start with a Haskell version. euler2 = (sum . filter even . takeWhile (<= 4000000)) fibs where fibs = 1 : 1 : zipWith (+) fibs (tail fibs) On 29/03/2011, at 9:33 PM, JohnyTex wrote: A "lazy" fib generator, with "lazy" in scare quotes because repeated evaluation will redo work, a version of lists:takewhile/2 adapted to "lazy" lists, his own copies of lists:sum/1 and lists:filter/2, and something rather like the Haskell version. There are many ways we can judge a piece of code. "Does it do the job?" Yes, it does. "Is it easy to see that it does the job?" Yes, it is. "Does it use the language idiomatically?" No. JohnyTex's Erlang code is basically an adaptation to Erlang of an approach which is idiomatic in Haskell. It has as its core idea "store a data structure representing all the Fibonacci numbers". Here's an Erlang version: euler2() -> fib_sum(1, 1, 4000000, 0). fib_sum(F0, F1, Limit, Sum) -> F2 = F0 + F1, if F2 > Limit -> Sum ; true -> fib_sum(F1, F2, Limit, case F2 band 1 of 0 -> Sum+F2 ; 1 -> Sum end) end. It has as its core idea "iterate over a prefix of the Fibonacci numbers". It allocates no storage and is tail recursive. It is, in fact, the *direct* equivalent of #include <stdio.h> int main(void) { int const Limit = 4000000; int Sum; int F0, F1, F2; Sum = 0; for (F0 = F1 = 1; (F2 = F1 + F0) <= Limit; F0 = F1, F1 = F2) if ((F2 & 1) == 0) Sum += F2; printf("%d\n", Sum); return 0; } In the C version and my Erlang version, "Is it easy to see that it does the job" gets the defensive answer "well, it may not be OBVIOUS, but ..." which really amounts to "no". Can we perhaps compromise? fold_fibs(Limit, Initial, Combiner) -> fold_fibs(0, 1, Limit, Initial, Combiner). fold_fibs(_, F1, Limit, State, _) when F1 > Limit -> State; fold_fibs(F0, F1, Limit, State, Combiner) -> fold_fibs(F1, F0+F1, Limit, Combiner(State,F1), Combiner). add_if_even(X, Y) when Y band 1 =:= 0 -> X+Y; add_if_even(X, _) -> X. euler2() -> fold_fibs(4000000, 0, fun add_if_even/2). This factorises the problem as "folding over a prefix of the Fibonacci numbers" and "summing even numbers". It isn't as immediately obvious as the Haskell version, but it should be easier to understand than the "bare bones" version. clear From nem@REDACTED Wed Mar 30 03:19:59 2011 From: nem@REDACTED (Geoff Cant) Date: Tue, 29 Mar 2011 18:19:59 -0700 Subject: [erlang-questions] gen:call({global, Name}, ...) Message-ID: <m1fwq5s8u8.fsf@erlang.geek.nz> Hi all, I discovered today that gen:call({global, Name}, Label, Request, Timeout) calls global:safe_whereis_name(Name) to determine the Pid to look up globally registered names. global:safe_whereis_name/1 doesn't seem to offer any particular safety and more importantly, serializes all global name lookups on a node. (Using global:whereis_name/1 instead is just an ets lookup). Can we safely make a change like https://github.com/archaelus/otp/commit/4f6e8a147b3c600eef2dd05f8ce0d51cf9c35383 in gen.erl and improve call time and reduce the load on global_name_server at a stroke? This git repo contains the patch I'm thinking of: git fetch git://github.com/archaelus/otp.git gen_where Cheers, -- Geoff Cant From wilsoncj1@REDACTED Wed Mar 30 05:20:54 2011 From: wilsoncj1@REDACTED (Chris Wilson) Date: Tue, 29 Mar 2011 23:20:54 -0400 Subject: [erlang-questions] Nested Supervisors Message-ID: <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> I'm looking for some examples on starting more than one level of supervisors in a supervision tree. Almost everything I've seen has been one supervisor and many workers and I've not seen any documentation or examples on how I might start multiple supervisors within one supervision tree. I'm trying to build a structure similar to what is below, a simple monitoring application to track events in a typical network environment within a given site. A site level supervisor handles N number of L3 networks and their workers and spawns N number of supervisors for VLANs in a given MST instance. This example is slightly contrived in that I "could" flatten this structure out, but I'm looking to do other things with those vlan workers and may go another level deeper. vlan_monitor_worker / vlan801_sup / net1_sup / \ / \ net1_mon_worker Site_sup net2_mon_worker \ / \ / net2_sup \ vlan802_sup \ vlan_monitor_worker Thanks in advance for any guidance. -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/e8491f2d/attachment.htm> From silent_vendetta@REDACTED Wed Mar 30 07:23:28 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Tue, 29 Mar 2011 22:23:28 -0700 Subject: [erlang-questions] Re: Nested Supervisors In-Reply-To: <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> References: <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> Message-ID: <SNT123-w3029EC5F18CE430304DAF884BC0@phx.gbl> Take a look at: http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles While it isn't specifically addressing your issue, the article does show how to nest supervisors. They do throw a few bits of functionality that could be put into separate modules into one, but it shouldn't be that hard to figure out at all. -Chris Hicks Date: Tue, 29 Mar 2011 23:20:54 -0400 From: wilsoncj1@REDACTED To: erlang-questions@REDACTED Subject: [erlang-questions] Nested Supervisors I'm looking for some examples on starting more than one level of supervisors in a supervision tree. ?Almost everything I've seen has been one supervisor and many workers and I've not seen any documentation or examples on how I might start multiple supervisors within one supervision tree. I'm trying to build a structure similar to what is below, a simple monitoring application to track events in a typical network environment within a given site. ?A site level supervisor handles N number of L3 networks and their workers and spawns N number of supervisors for VLANs in a given MST instance. ?This example is slightly contrived in that I "could" flatten this structure out, but I'm looking to do other things with those vlan workers and may go another level deeper. ? ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?vlan_monitor_worker?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /?? ? ? ? ? ? ? ? ? ? ? vlan801_sup?? ? ? ? ? ? ? ? ? ? /?? ? ? ? ?net1_sup ?? ? ? ? / ? ? ? ? ? \?? ? ? ?/ ? ? ? ? ? ? \?? ? ? ? ? ? ? ? ? ? ? net1_mon_worker?Site_sup??? ? ? ? ? ? ? ? ? ? ? net2_mon_worker?? ? ? ?\ ? ? ? ? ? ??/ ?? ? ? ? \ ? ? ? ? ? /?? ? ? ? ?net2_sup?? ? ? ? ? ? ? ? ? ? \?? ? ? ? ? ? ? ? ? ? ?vlan802_sup?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? \?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?vlan_monitor_worker Thanks in advance for any guidance. -Chris _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110329/ac6ef343/attachment.htm> From g9414002.pccu.edu.tw@REDACTED Wed Mar 30 08:56:31 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Wed, 30 Mar 2011 14:56:31 +0800 Subject: [erlang-questions] Re: Newbie Erlang programmer looking for feedback! :) In-Reply-To: <c4415989-af65-4fb5-bde1-ff93738e9155@f2g2000yqf.googlegroups.com> References: <c4415989-af65-4fb5-bde1-ff93738e9155@f2g2000yqf.googlegroups.com> Message-ID: <AANLkTik_Z=wEGEcfyXTnxFqoAYLfg-j44enXJznb2a2z@mail.gmail.com> On Tue, Mar 29, 2011 at 4:33 PM, JohnyTex <ekeroth.christoffer@REDACTED> wrote: > Anyway, here's my solution to the second problem from Project Euler > (http://projecteuler.net/index.php?section=problems&id=2). The problem > is to find the sum of all the even fibonacci numbers below four > million; I thought this could be a good exercise since it would let me > try my hand at implementing lazy sequences and working with lists. > > I split up the program into two modules. The first one is for handling > lazy sequences, since I thought that would be a good way to find our > list of primes: > cut(). > %% Lazy sequence that generates fibonacci numbers > lazyfib(A, B) -> [A | fun () -> lazyfib(B, A + B) end]. > lazyfib() -> lazyfib(0, 1). > > > %% Generate all fibonacci terms that are less than 4 million and sum > the > %% even terms > solve() -> > Fibs = seqs:takewhile(fun (X) -> X < 4000000 end, lazyfib()), > sum(filter(fun (X) -> X rem 2 =:= 0 end, Fibs)). > > Thanks in advance, and please tell me if this is not the appropriate > forum for this kind of question! :) > It helps nothing but only generating a Fibonacci number sequence. The lazyfib/2 generate a lazy sequence and give the whole to seqs:takewhile/2, which is dedicated to the lazy sequence. IMHO, first I have a fib/1 function as interface with its implementation fibonacci/4. -module(test). -compile(export_all). fib(0) -> 1; fib(1) -> 1; fib(N) -> fibonacci(0, N, fib(0), fib(1)). fibonacci(N, T, N_2, _) when N == T -> N_2; fibonacci(N, _, _, _) when N < 0 -> false; fibonacci(N, T, N_2, N_1) -> fibonacci(N + 1, T, N_1, N_2 + N_1). I'd better to use this native fib/1 function to generate the Fibonacci number list. In fibonacci/4 I knew two facts: 1. By calling fibonacci(N, N, N_2, N_1), I'll get any fib(N) if N_2 and N_1 are correct base numbers. 2. For the next number in the sequence, { N, N_2, N_1 } are key variants. For any number with variants { N, N_2, N_1 }, it should pass { N+1, N_1, N_2 + N_1 } to its next number. So, I wrote fib_seq/2 as interface to generate a limited sequence of Fibonacci numbers, which has a lazy version of implementation fib_seq_l/4. fib_seq(From, Limited_value) when From < 0 -> fib_seq(0, Limited_value); fib_seq(From, Limited_value) -> fib_seq_l(From, fib(From), fib(From+1), Limited_value). fib_seq_l(_, N_2, _, Limited_value) when N_2 > Limited_value -> []; fib_seq_l(N, N_2, N_1, Limited_value) -> [ fibonacci(N, N, N_2, N_1) | fun() -> fib_seq_l(N + 1, N_1, N_2 + N_1, Limited_value) end ]. Now it can generate a Fibonacci number sequence limited to a maximum value 4000000. > *test:fib_seq(0, 4000000).* [1|#Fun<test.0.28629712>] The result of a sequence like test:fib_seq(0, 4000000) can be exploded by explode/1. explode([]) -> []; explode(L) when is_function(L) -> explode(L()); explode([H|T]) -> [H|explode(T)]. > *test:explode(test:fib_seq(0,4000000)).* [1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,1597,2584, 4181,6765,10946,17711,28657,46368,75025,121393,196418, 317811,514229|...] And even_seq_l takes another problem, picking all even numbers from a maybe-with-infinite-length list of numbers. even_seq_l([]) -> []; even_seq_l([H|T]) when H rem 2 == 0 -> case is_function(T) of true -> [ H | fun() -> even_seq_l(T()) end ]; false -> [ H | fun() -> even_seq_l(T) end ] end; even_seq_l([_|T]) -> case is_function(T) of true -> fun() -> even_seq_l(T()) end; false -> fun() -> even_seq_l(T) end end. And then, by merging even_seq_l/1 and fib_seq/2 functions, I can get even numbers from a limited list of Fibonacci numbers. even_fib_seq(From, Limited_value) -> even_seq_l(fib_seq(From, Limited_value)). It's almost there. solution_problem2() -> R = even_fib_seq(0, 4000000), io:format("Sum of ~w is ~w~n", [explode(R), sum_l(R)]). And the result should be > *test:solution_problem2().* Sum of [2,8,34,144,610,2584,10946,46368,196418,832040,3524578] is 4613732 ok I need sum_l/2 for lazy summing. sum_l([]) -> 0; sum_l(L) when is_function(L) -> sum_l(L()); sum_l([H|T]) -> H + sum_l(T). However the +/2 operator slightly break the laziness if sum_l/1 tend to accept a big sequence. /yau -- Best Regards. --- Y-H. H. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/cde0511d/attachment.htm> From per.melin@REDACTED Wed Mar 30 10:40:47 2011 From: per.melin@REDACTED (Per Melin) Date: Wed, 30 Mar 2011 10:40:47 +0200 Subject: [erlang-questions] Re: [erlang-questions 71] Erlang release generation In-Reply-To: <AANLkTim1H6KFjn=8NGC=EispZ=qeBxQ2RJkZfWSEoovN@mail.gmail.com> References: <4D902590.8060409@m5net.com> <AANLkTimb39mFr6mApF0En=UJsi=ONndJ=7AyVghAQpxG@mail.gmail.com> <AANLkTim0H4KwjKVe9mLzAzAhmdMxr7jiueea1uGoiB3k@mail.gmail.com> <AANLkTinZw007nBx4zxVhcypSD3p9nV3MKcV=rUKrVCqN@mail.gmail.com> <AANLkTim1H6KFjn=8NGC=EispZ=qeBxQ2RJkZfWSEoovN@mail.gmail.com> Message-ID: <AANLkTinQzPk6wJR--oZ-nNbVLLzFMMTv7=iUo2Le3akx@mail.gmail.com> On Wed, Mar 30, 2011 at 1:13 AM, Sam Bobroff <sam@REDACTED> wrote: > 2011/3/29 Per Melin <per.melin@REDACTED> >> 2. start.boot. >> For the same release you get a boot file called foo.boot. But >> release_handler will barf unless there is a start.boot (which I >> believe is really irrelevant unless you run in embedded mode with the >> default scripts). So maybe when you include the option {boot_rel, >> "foo"} that could mean that foo.boot will instead be called >> start.boot? > > Yes. I've successfully used boot_rel to specify my own boot script. It works > fine and the specified script is the one that runs if you don't provide one > on the command line. > Peace, > Sam. I was wrong, kind of. You are correct, you *do* get a start.boot, in the bin directory. I never saw that before since I usually skip that directory completely (with the reltool option {excl_sys_filters, ["^bin/"]}) and use my own start script instead. But if you want to do proper upgrades with release_handler, you'll need to use the copy of the boot file that was created for you in the releases/<vsn> directory instead, since each version of the release has its own boot file. And, as far as I know, release_handler:unpack_release/1 will not update your bin/start.boot with subsequent releases, unless you are running in embedded static emulator diskless mode. It is quite possible that I've got some part of this wrong. When it comes to Erlang release handling, I regularly find that to be the case. From raimo+erlang-questions@REDACTED Wed Mar 30 11:14:33 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 30 Mar 2011 11:14:33 +0200 Subject: [erlang-questions] Re: [erlang-questions 10] Re: Dead links on new site In-Reply-To: <20110325183303.GA10692@erix.ericsson.se> References: <AANLkTi=dHtSPCDL+WLb83XffKPcbEyC7HCf+fhaG=os4@mail.gmail.com> <20110325183303.GA10692@erix.ericsson.se> Message-ID: <20110330091433.GA24025@erix.ericsson.se> On Fri, Mar 25, 2011 at 07:33:03PM +0100, Raimo Niskanen wrote: > On Fri, Mar 25, 2011 at 02:16:38PM -0400, Jordan Wilberding wrote: > > I couldn't immediately find a contact for the erlang.org website, so just > > going to post here. > > > > There is a dead link for "entire source > > code<http://www.erlang.org/downloads.html>" and > > "EPL translation <http://www.erlang.org/license/EPL1x0-explained.html>"at > > http://www.erlang.org/about.html Now they are fixed. Contact info maybe too brief, but should be human parsable. > > > > Thanks! > > Jordan Wilberding > > The de-facto standard webmaster@REDACTED should work, but I will > add a contacts link somewhere, and check your findings... > Thank you for reporting this! > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From pekadan@REDACTED Wed Mar 30 11:32:41 2011 From: pekadan@REDACTED (Peter Andersson) Date: Wed, 30 Mar 2011 11:32:41 +0200 Subject: [erlang-questions] Re: Common Test Question In-Reply-To: <BANLkTi==BeCsVvr_py-RX_28i0=CRzZnqg@mail.gmail.com> References: <BANLkTi=g8rsUgv-tZUw1bALiVd5BWPViaQ@mail.gmail.com> <AANLkTimiHfYK8g8AMFo9PgJJfzG2CC90X0Af+ARrV3_d@mail.gmail.com> <BANLkTi==BeCsVvr_py-RX_28i0=CRzZnqg@mail.gmail.com> Message-ID: <AANLkTikEQiMT9paeJJJ=bAjahQyH7U+bWgi6NMT-UZoc@mail.gmail.com> Thanks for the input Jordan. I have been trying to come up with a good reason for letting init- and end_per_group execute even for an empty group, but can't really see that being a useful feature. So I will probably let Common Test ignore it, like you suggest. Best, Peter Ericsson AB, Erlang/OTP 2011/3/29 Jordan West <jwest@REDACTED>: > Peter, > Thanks for letting me know. I considered it a bug as well. On the topic of > the config functions, I think other testing frameworks like Ruby's RSpec > ignore them when no tests in the group exist. I assume this is to prevent > hard to debug side-effects in tests and as a small optimization. > Jordan West > On Tue, Mar 29, 2011 at 1:56 AM, Peter Andersson <pekadan@REDACTED> wrote: >> >> Hi Jordan, >> >> I learned about this problem just the other day. I consider it a bug >> really. An empty group should be ignored, or possibly, the config >> functions for the group should be executed (even if there's no test >> case to configure for). I have written a ticket on it and a fix will >> come shortly. >> >> Thanks for reporting! >> >> Best regards, >> Peter >> >> Ericsson AB, Erlang/OTP >> >> 2011/3/29 Jordan West <jwest@REDACTED>: >> > I've recently been using common_test on several new projects and am >> > really >> > liking it. I ran across something today I found odd, however, and wanted >> > to >> > know if it was the expected behavior. ?I was expecting that if I have a >> > group with no tests that group should, essentially, be ignored. This >> > seems >> > to be the case if the group is contained in another but not if the group >> > is >> > an element in the list returned by all/0. In the latter case the suite >> > errors out with "Error in suite detected: 'Invalid reference to group >> > [GROUP >> > NAME] in [SUITE NAME]:all/0'". >> > Is this intentional and if so, why are sub-groups treated differently? >> > Is >> > there a reason its not a warning instead? For the most part I'm just >> > curious. I do like setting up my groups sometimes ahead of writing tests >> > and >> > ran into this when I tried to leave a top-level group empty for a short >> > while. >> > A minimal code example is below. I'm using R14B02 on OS-X with >> > common_test >> > 1.5.3 (I installed today off the Erlang Factory USB stick, thanks for a >> > great conference everyone). Oh, and this is my first message to the >> > list. >> > Hello, all. >> > %% a.erl >> > -module(a). >> > -compile(export_all). >> > my_fun() -> >> > ? ? 1. >> > %% a_SUITE.erl >> > -module(a_SUITE). >> > -compile(export_all). >> > groups() -> >> > ? ? [{group_1, >> > ? ? ? [shuffle], >> > ? ? ? [test_1, {group, group_3}]}, >> > ? ? ?{group_2, >> > ? ? ? [shuffle], >> > ? ? ? []}, % this empty test list causes issues if empty >> > ? ? ?{group_3, >> > ? ? ? [],[]}]. % this empty test list seems to be able to be empty >> > all() -> >> > ? ? [ >> > ? ? ?{group, group_1}, >> > ? ? ?{group, group_2} %% If this is remove the suite won't error out >> > ? ? ]. >> > test_1(_) -> >> > ? ? 1 = a:my_fun(). >> >>> ?run_test -suite a_SUITE.erl >> > ... >> > Error in suite detected: 'Invalid reference to group group_2 in >> > a_SUITE:all/0' >> > Jordan West >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > From max.lapshin@REDACTED Wed Mar 30 12:55:22 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 30 Mar 2011 14:55:22 +0400 Subject: [erlang-questions] Re: Upgrading a gen_tcp acceptor process In-Reply-To: <1301381281.5251.27.camel@seasc1137> References: <4D91751E.5040508@m5net.com> <1301381281.5251.27.camel@seasc1137> Message-ID: <AANLkTik4L7OwLLck13mJHToEouLzizwpyqkvgF7vnhM=@mail.gmail.com> What a problem? http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles From olivier.boudeville@REDACTED Wed Mar 30 13:29:53 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Wed, 30 Mar 2011 13:29:53 +0200 Subject: [erlang-questions] List comprehension questions Message-ID: <OF73F2F9C4.4B183BD4-ONC1257863.003747FA-C1257863.003F28EF@notes.edfgdf.fr> Hi, Reading http://www.erlang.org/doc/reference_manual/expressions.html#id77052 it is not obvious that the order induced by a single generator is necessarily preserved by the comprehension, as nevertheless suggested by all experiments like: 1> [ X || X <- [1,2,3] ]. [1,2,3] I suppose this order preservation is true. Then, I imagine that, internally, list comprehensions are translated to code very similar to the one that would be produced if the developer had written an appropriate accumulator-based tail-recursive function? If so, does it imply that using a list comprehension on a semantically unordered set of elements will involve a useless reversal (like lists:reverse/1) of the resulting list? Thanks in advance for any answer, Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/66b52053/attachment.htm> From raimo+erlang-questions@REDACTED Wed Mar 30 13:41:39 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 30 Mar 2011 13:41:39 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> Message-ID: <20110330114139.GA28211@erix.ericsson.se> On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: > Cacheability ? > > I can reload the page multiples times and still download images. > > It seems also that images don't have any 'cache' attributes. > (I don't recommended using ETags unless you're absolutely sure that > ETags will be always the > same on multiple web servers. i.e. some webservers use inode > information to compute the ETag > which is different on different servers...thought I don't know how > erlang-web compute it) Please inform a newbie, I just checked HTML4.01 and did not find any "cache" attribute. > > Furthermore all links in the 'Articles' part seems dead > (http://www.erlang.org/doc.html ) Can you describe more thoroughly. I find no dead links. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From gleber.p@REDACTED Wed Mar 30 13:46:38 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 30 Mar 2011 13:46:38 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <20110330114139.GA28211@erix.ericsson.se> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> Message-ID: <AANLkTi=V0fs12Fpvw5SaTE4FmKmzaHovSz+tCms0kmWp@mail.gmail.com> On Wed, Mar 30, 2011 at 13:41, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote: > Please inform a newbie, I just checked HTML4.01 and did not > find any "cache" attribute. I think Koener meant HTTP caching. Here's a random but nice looking article about it: http://betterexplained.com/articles/how-to-optimize-your-site-with-http-caching/ From vinoski@REDACTED Wed Mar 30 13:50:47 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 30 Mar 2011 07:50:47 -0400 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <20110330114139.GA28211@erix.ericsson.se> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> Message-ID: <AANLkTikZoHLuUL4y1WJ7S91eWFp5-iiu2-X8oYU9tAuO@mail.gmail.com> On Wed, Mar 30, 2011 at 7:41 AM, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote: > On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: >> Cacheability ? >> >> I can reload the page multiples times and still download images. >> >> It seems also that images don't have any 'cache' attributes. >> (I don't recommended using ETags unless you're absolutely sure that >> ETags will be always the >> same on multiple web servers. i.e. some webservers use inode >> information to compute the ETag >> which is different on different servers...thought I don't know how >> erlang-web compute it) > > Please inform a newbie, I just checked HTML4.01 and did not > find any "cache" attribute. He's referring to cache control headers such as Expires, ETag, and Cache-Control. These aren't part of HTML but rather are typically configured for the web server itself. See section 13 of the HTTP 1.1 spec: http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13 --steve From fred.hebert@REDACTED Wed Mar 30 13:58:14 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Wed, 30 Mar 2011 07:58:14 -0400 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <20110330114139.GA28211@erix.ericsson.se> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> Message-ID: <354F7907-29F9-4C33-BE13-AFA666313B42@erlang-solutions.com> On 2011-03-30, at 07:41 AM, Raimo Niskanen wrote: > On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: >> >> Furthermore all links in the 'Articles' part seems dead >> (http://www.erlang.org/doc.html ) > > Can you describe more thoroughly. I find no dead links. All the 'Articles' on Ericsson's website lead to a dead page telling you they are not available anymore. Same for the CSLab publications. > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Fred H?bert http://www.erlang-solutions.com From koops.j@REDACTED Wed Mar 30 14:11:59 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Wed, 30 Mar 2011 14:11:59 +0200 Subject: [erlang-questions] Distributed application takeover Message-ID: <AANLkTi=RuSSaeR_w4TLFJeKW6-0DhfQ5a78RCZ1Uq668@mail.gmail.com> Hi, I'm trying to find out how the distributed application controller works internally. I'm especially interested in the implementation of an application takeover. In case an application runs on node A, and is taken over by node B, what should happen is that it is first started on node B, so that there are two instances of the application running simultaneously for a brief period of time, and then stopped on node A. However, I cannot figure out where this stopping happens in dist_ac.erl. If I understand correctly, this should happen in response to a ac_application_run message from the application_controller. This message is received by the dist_ac on node B, and a dist_ac_app_started message is then broadcast to the dist_acs on all connected nodes. The dist_ac of node A receives this message, notices that the application is still running locally, and decides to shut down the application on its own node -- at least that is what the comments say (dist_ac.erl, line 529): %% Another node tookover from me; stop my application %% and update the running list. But all I can see is that the dist_ac's list of applications is updated to indicate that the application is no longer running locally -- I cannot find where the application_controller is instructed to actually shutdown the application. Can anyone point me in the right direction? Thanks, Jeroen -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/488f405d/attachment.htm> From raimo+erlang-questions@REDACTED Wed Mar 30 14:21:08 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 30 Mar 2011 14:21:08 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <354F7907-29F9-4C33-BE13-AFA666313B42@erlang-solutions.com> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> <354F7907-29F9-4C33-BE13-AFA666313B42@erlang-solutions.com> Message-ID: <20110330122108.GA31791@erix.ericsson.se> On Wed, Mar 30, 2011 at 07:58:14AM -0400, Fr?d?ric Trottier-H?bert wrote: > > On 2011-03-30, at 07:41 AM, Raimo Niskanen wrote: > > > On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: > > >> > >> Furthermore all links in the 'Articles' part seems dead > >> (http://www.erlang.org/doc.html ) > > > > Can you describe more thoroughly. I find no dead links. > > > All the 'Articles' on Ericsson's website lead to a dead page telling you they are not available anymore. Same for the CSLab publications. Allrigh, I am sorry, but it was not obvious to me. I may be tired. I tried all links in the toplevel section 'articles'... But, as you say, on the page http://www.erlang.org/doc.html, all links under the heading 'Articles' are dead. I have searched the Ericsson Review's archives and they just go back 10 years, i.e to 1999, and all our articles are from 1997 and 1998. We probably will have to remove these links... > > > > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Fred H?bert > http://www.erlang-solutions.com > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlangsiri@REDACTED Wed Mar 30 14:54:23 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 30 Mar 2011 14:54:23 +0200 Subject: [erlang-questions] Re: rb help error In-Reply-To: <AANLkTikJo0y3ot+QYvYGSbrkg2EqEu+F0YyZ9tHz=JAf@mail.gmail.com> References: <AANLkTikJo0y3ot+QYvYGSbrkg2EqEu+F0YyZ9tHz=JAf@mail.gmail.com> Message-ID: <AANLkTinqCWDY5bzAyy7kN0nQ7H-P735dmYvfqDJ_tUvF@mail.gmail.com> Hi Bob - Thanks for pointing this out! I will correct it. Regards /siri 2011/3/29 Bohuslav Svancara <bsvancara@REDACTED> > Wrong help in rb: > > rb:filter(Filters, Dates) - > same as rb:filter/1 but accepts date ranges to filter reports. > Dates must be of the following form: > - {StartDate, EndDate} > StartDate = EndDate = {{Y-M-D},{H,M,S}} > > > Should be: > StartDate = EndDate = {{Y,M,D},{H,M,S}} > > Bob > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/3c0219f0/attachment.htm> From ulf.wiger@REDACTED Wed Mar 30 16:00:42 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 30 Mar 2011 16:00:42 +0200 Subject: [erlang-questions] Re: Distributed application takeover In-Reply-To: <AANLkTi=RuSSaeR_w4TLFJeKW6-0DhfQ5a78RCZ1Uq668@mail.gmail.com> References: <AANLkTi=RuSSaeR_w4TLFJeKW6-0DhfQ5a78RCZ1Uq668@mail.gmail.com> Message-ID: <0DBAD7AB-FE6B-4A54-A07B-C8ABAB6E8ACF@erlang-solutions.com> Hi Jeroen, This is a bit tricky, and can be very confusing for someone trying to build their own cluster controller. :) The stopping of the application is done by the local application_controller when it is told that the application is running somewhere else. In other words, dist_ac kindly forwards the information to the local AC, and it is the local AC that in this particular case takes responsibility for 'knowing' that the local instance should be stopped. See application_controller:handle_application_started/3 https://github.com/erlang/otp/blob/OTP_R14B02/lib/kernel/src/application_controller.erl#L913 (The dirty deed actually happens on line 946). The reason I know this, is that in the AXD 301, we ran multiple instances of some applications, distributed across several mated pairs - each instance having its own standby node. This can be done by writing an own distributed AC, but it has to be smart enough to know _when_ to forward the {started, Node} status to the local AC; if an instance was running locally - and _was supposed to_ do so (i.e. not involved in a takeover), the distributed AC had to suppress this message. BR, Ulf W On 30 Mar 2011, at 14:11, Jeroen Koops wrote: > Hi, > > I'm trying to find out how the distributed application controller works internally. I'm especially interested in the implementation of an application takeover. > > In case an application runs on node A, and is taken over by node B, what should happen is that it is first started on node B, so that there are two instances of the application running simultaneously for a brief period of time, and then stopped on node A. > > However, I cannot figure out where this stopping happens in dist_ac.erl. If I understand correctly, this should happen in response to a ac_application_run message from the application_controller. This message is received by the dist_ac on node B, and a dist_ac_app_started message is then broadcast to the dist_acs on all connected nodes. The dist_ac of node A receives this message, notices that the application is still running locally, and decides to shut down the application on its own node -- at least that is what the comments say (dist_ac.erl, line 529): > > %% Another node tookover from me; stop my application > %% and update the running list. > > But all I can see is that the dist_ac's list of applications is updated to indicate that the application is no longer running locally -- I cannot find where the application_controller is instructed to actually shutdown the application. > > Can anyone point me in the right direction? > > Thanks, > > Jeroen _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/0e3aed1c/attachment.htm> From norton@REDACTED Wed Mar 30 18:15:36 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Thu, 31 Mar 2011 01:15:36 +0900 Subject: [erlang-questions] erl_driver - is it safe to call driver_output directly by an async thread? Message-ID: <CE215CCE-0F83-4F67-B5D8-4E452325BA4F@lovely.email.ne.jp> Hello. I have a question regarding erl_driver and the async thread pool. Is it safe to call driver_output directly by an async thread? From my limited testing, it works well. From the documentation, I'm not sure. thanks, - Joe N. http://www3.erlang.org/documentation/doc-5.5.5/erts-5.5.5/doc/html/erl_driver.html long driver_async (ErlDrvPort port, unsigned int* key, void (*async_invoke)(void*), void* async_data, void (*async_free)(void*)) This function performs an asynchronous call. The function async_invoke is invoked in a thread separate from the emulator thread. This enables the driver to perform time-consuming, blocking operations without blocking the emulator. int driver_output(ErlDrvPort port, char *buf, int len) The driver_output function is used to send data from the driver up to the emulator. The data will be received as terms or binary data, depending on how the driver port was opened. The data is queued in the port owner process' message queue. Note that this does not yield to the emulator. (Since the driver and the emulator runs in the same thread.) The parameter buf points to the data to send, and len is the number of bytes. The return value for all output functions is 0. (Unless the driver is used for distribution, in which case it can fail and return -1. For normal use, the output function always returns 0.) Joseph Wayne Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/580148a2/attachment.htm> From watson.timothy@REDACTED Wed Mar 30 19:15:53 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 30 Mar 2011 18:15:53 +0100 Subject: [erlang-questions] Re: erl_driver - is it safe to call driver_output directly by an async thread? In-Reply-To: <CE215CCE-0F83-4F67-B5D8-4E452325BA4F@lovely.email.ne.jp> References: <CE215CCE-0F83-4F67-B5D8-4E452325BA4F@lovely.email.ne.jp> Message-ID: <AANLkTikyDRskGfdvokzzaKah0=GK4z8a48dHJ2VNTAuK@mail.gmail.com> On 30 March 2011 17:15, Joseph Wayne Norton <norton@REDACTED> wrote: > Hello. > > I have a question regarding erl_driver and the async thread pool. > > Is it safe to call driver_output directly by an async thread? > All the erl_driver functions are documented as "thread-safe", sometimes with the caveat "when the emulator with SMP support is used." Unless it's stated in the docs, it's probably not safe to call them from an async thread without explicit synchronisation. I'm sure one of the OTP team will dive in and say if that assumption is wrong. From bob@REDACTED Wed Mar 30 19:22:07 2011 From: bob@REDACTED (Bob Cowdery) Date: Wed, 30 Mar 2011 18:22:07 +0100 Subject: [erlang-questions] linked-in driver Message-ID: <4D9366BF.6080106@bobcowdery.plus.com> I've done this before and more or less borrowed the code. However, it was 4 years ago and clearly I've forgotten something important. In essence I'm sending something like this to a linked-in driver. term_to_binary({do_dsp, <<some binary data>>}) I've checked what this gives me in the shell using just <<0>> as the binary. I've checked that exactly the same thing arrives at this point in the C program. ei_decode_tuple_header(rbuf->buff, &rbuf->index, &arity); where: ei_x_buff rbuf; static void dttsp_port_drv_output(ErlDrvData handle, char *buff, int bufflen) { rbuf.buff = buff; rbuf.buffsz = bufflen; rbuf.index = 0; ... } When I run it I get arity of 0 when it should be 2. What am I doing wrong? Bob From comptekki@REDACTED Wed Mar 30 19:47:13 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 30 Mar 2011 11:47:13 -0600 Subject: [erlang-questions] supervisors Message-ID: <AANLkTik8_FmG1hipmUkmD2oKgS36XPKsee-P52Hsgz2q@mail.gmail.com> when I start and app that works fine: =PROGRESS REPORT==== 30-Mar-2011::11:28:05 === supervisor: {local,eapp_sup} started: [{pid,<0.70.0>}, {name,{a_sup,a1}}, {mfargs,{a_sup,start_link,[a1]}}, {restart_type,permanent}, {shutdown,brutal_kill}, {child_type,supervisor}] In eapp_sup I have: start_a(Module, Name) -> supervisor:start_child(?MODULE, child(Module, Name)). child(Module, Name) -> {{Module, Name}, {Module, start_link, [Name]}, permanent, brutal_kill, supervisor, [Name]}. from starting above it would seem that a_sup has the name a1, but if you run appmon it shows just as a_sup. When I then run eapp:start_a(a_sup,a2). I would assume that a_sup could load as a second supervisor process with name a2, but this is what I get: eapp_sup:start_a(a_sup,a2). {error,{already_started,<0.70.0>}} I then create a_sup2 and eapp_sup:start_a(a_sup2, a1) and that loads fine, since the module "name" is not the same in the system as a_sup. But it seems like a name should be assigned to these to make them unique by name and be able to use the same supervisor code to supervise some more children. Is that possible? thx, -wes From fritchie@REDACTED Wed Mar 30 20:16:51 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 30 Mar 2011 13:16:51 -0500 Subject: [erlang-questions] Re: Nested Supervisors In-Reply-To: Message of "Tue, 29 Mar 2011 23:20:54 EDT." <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> Message-ID: <35919.1301509011@snookles.snookles.com> Chris, sub-supervisors can be nested arbitrarily deep, if that's the structure you need. The deepest process tree that I could think of, off the top of my head (aside from the OTP 'kernel' app), is from an old Ericsson elevators demo/training app. A screen shot is at: http://www.snookles.com/scotttmp/elevators-appmon-view.png -Scott From dale@REDACTED Wed Mar 30 20:47:04 2011 From: dale@REDACTED (Dale Harvey) Date: Wed, 30 Mar 2011 19:47:04 +0100 Subject: [erlang-questions] Programatically Starting Erlang Nodes Message-ID: <AANLkTinMr+Ce7dJUrs-6V_29mL64OiRfWqhZrFaJ4GBa@mail.gmail.com> How do people manage programatically starting and stopping seperate erlang distributed nodes? I would really prefer the option to do this inside erlang, making starting a node as easy as node:start('hello@REDACTED'), test_server start node uses a port under the hood which leaves the child node closely tied to the parent node, starting -detached nodes has the opposite problem of being very early to leave dangling nodes whenever they arent wanted. so the basic things I am looking for easily create a node from within erlang be able to to detach from child node and leave it running when parent node exits a nice way to ensure the stopping child nodes actually shutdown, even when erlang distribution falls down. Can anyone think of elegant examples of setting up / tearing down distributed erlang nodes? one managed outside erlang are also suitable Cheers Dale From mevans@REDACTED Wed Mar 30 21:04:25 2011 From: mevans@REDACTED (Evans, Matthew) Date: Wed, 30 Mar 2011 15:04:25 -0400 Subject: [erlang-questions] Re: Programatically Starting Erlang Nodes In-Reply-To: <AANLkTinMr+Ce7dJUrs-6V_29mL64OiRfWqhZrFaJ4GBa@mail.gmail.com> References: <AANLkTinMr+Ce7dJUrs-6V_29mL64OiRfWqhZrFaJ4GBa@mail.gmail.com> Message-ID: <B293D35E9E20694A8D9DFFE1BC655C556AAACFAB44@vvexch.verivue.com> Perhaps you should look at what these guys did: https://support.process-one.net/doc/display/ERL/Starting+a+set+of+Erlang+cluster+nodes ________________________________________ From: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED] On Behalf Of Dale Harvey [dale@REDACTED] Sent: Wednesday, March 30, 2011 2:47 PM To: erlang-questions Subject: [erlang-questions] Programatically Starting Erlang Nodes How do people manage programatically starting and stopping seperate erlang distributed nodes? I would really prefer the option to do this inside erlang, making starting a node as easy as node:start('hello@REDACTED'), test_server start node uses a port under the hood which leaves the child node closely tied to the parent node, starting -detached nodes has the opposite problem of being very early to leave dangling nodes whenever they arent wanted. so the basic things I am looking for easily create a node from within erlang be able to to detach from child node and leave it running when parent node exits a nice way to ensure the stopping child nodes actually shutdown, even when erlang distribution falls down. Can anyone think of elegant examples of setting up / tearing down distributed erlang nodes? one managed outside erlang are also suitable Cheers Dale _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From jack@REDACTED Wed Mar 30 21:37:17 2011 From: jack@REDACTED (Jack Moffitt) Date: Wed, 30 Mar 2011 13:37:17 -0600 Subject: [erlang-questions] [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend Message-ID: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> Hi all, I just wanted to tell everyone that my iOS game, Snack Words, finally launched a week or so ago. I'm a pretty big fan of word games, and I was really surprised there are so few multiplayer games in the App Store. It seemed like a perfect job for Erlang too, and I gave a talk about it at recent Erlang Factory in San Francisco. The full server implementation, including rating system, pairing, and robot players clocks in at ~2,400 lines of code, including comments and whitespace. There's another 900 lines of tests. It really amazed me at the end how reading the main client gen_fsm code is basically just a funnily formated description of the protocol itself. Perhaps I'll be able to write the client side of future games in Erlang soon :) Hope you enjoy! jack. From torben.lehoff@REDACTED Wed Mar 30 22:02:56 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 30 Mar 2011 22:02:56 +0200 Subject: [erlang-questions] Anyone interested in an Neotoma based wikicreole parser? Message-ID: <AANLkTinANCj9h7vfix8KUkDaZSdeGy6hvdY2UcepehRt@mail.gmail.com> Hi, I have a fairly complete implementation of a wikicreole parser lying around which I could wrap up a bit and release as open source provided that there would be people willing to enhance and use it. I am not able to drive the project. wikicreole is a unification of the major wiki markup languages. The code is based on wikicreole 1.0 - needs some updates to reach the frozen wikicreole 1 specification - and Neotoma 1.4 (as far as I remember), so it needs a bit of love and affection. Any takers? Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110330/a6c0b9ee/attachment.htm> From kenneth.lundin@REDACTED Wed Mar 30 22:15:44 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 30 Mar 2011 22:15:44 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <20110330122108.GA31791@erix.ericsson.se> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> <354F7907-29F9-4C33-BE13-AFA666313B42@erlang-solutions.com> <20110330122108.GA31791@erix.ericsson.se> Message-ID: <AANLkTi=MRN46cBUENs=bz8pa3XAd2bqYR2QQiv=E7ybA@mail.gmail.com> Vi kan kanske l?gga upp egna kopior av dessa artiklar. Jag har dem p? en CD n?gonstans. /Kenneth On Wed, Mar 30, 2011 at 2:21 PM, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote: > On Wed, Mar 30, 2011 at 07:58:14AM -0400, Fr?d?ric Trottier-H?bert wrote: >> >> On 2011-03-30, at 07:41 AM, Raimo Niskanen wrote: >> >> > On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: >> >> >> >> >> Furthermore all links in the 'Articles' part seems dead >> >> (http://www.erlang.org/doc.html ) >> > >> > Can you describe more thoroughly. I find no dead links. >> >> >> All the 'Articles' on Ericsson's website lead to a dead page telling you they are not available anymore. Same for the CSLab publications. > > Allrigh, I am sorry, but it was not obvious to me. I may be tired. > I tried all links in the toplevel section 'articles'... > > But, as you say, on the page http://www.erlang.org/doc.html, > all links under the heading 'Articles' are dead. > > I have searched the Ericsson Review's archives and they just > go back 10 years, i.e to 1999, and all our articles are from > 1997 and 1998. We probably will have to remove these links... > >> >> > >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > -- >> > >> > / Raimo Niskanen, Erlang/OTP, Ericsson AB >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vances@REDACTED Wed Mar 30 23:19:38 2011 From: vances@REDACTED (Vance Shipley) Date: Wed, 30 Mar 2011 17:19:38 -0400 Subject: [erlang-questions] Re: Nested Supervisors In-Reply-To: <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> References: <AANLkTi=F_zWSQk1K+G9gxbZnacjXfP7T9k_4Be0UB3Xn@mail.gmail.com> Message-ID: <20110330211938.GM6861@h216-235-12-173.host.egate.net> On Tue, Mar 29, 2011 at 11:20:54PM -0400, Chris Wilson wrote: } I'm looking for some examples on starting more than one level of supervisors } in a supervision tree. Almost everything I've seen has been one supervisor } and many workers and I've not seen any documentation or examples on how I } might start multiple supervisors within one supervision tree. I make heavy use of supervision in all my applications. While it's been said that more than a couple levels of supervision becomes unwieldy I insist on every process being supervised. If for no other reason than that application:stop/1 should cleanly stop every process, or in a more practical sense `q().` from a shell prompt should exit silently and without hanging. My LAPD protocol implementation(*) uses three levels of supervision and is well documented: http://motivity.ca/lapd/ The fun comes in when you are dynamically starting and stopping processes in the tree. (*) http://code.google.com/p/lapderl/ -- -Vance From antoine.koener@REDACTED Thu Mar 31 00:07:55 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Thu, 31 Mar 2011 00:07:55 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <AANLkTikZoHLuUL4y1WJ7S91eWFp5-iiu2-X8oYU9tAuO@mail.gmail.com> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> <AANLkTikZoHLuUL4y1WJ7S91eWFp5-iiu2-X8oYU9tAuO@mail.gmail.com> Message-ID: <A6F075E2-36F4-450C-9118-34C9C5D700CB@gmail.com> On Mar 30, 2011, at 13:50 , Steve Vinoski wrote: > On Wed, Mar 30, 2011 at 7:41 AM, Raimo Niskanen > <raimo+erlang-questions@REDACTED> wrote: >> On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: >>> Cacheability ? >>> >>> I can reload the page multiples times and still download images. >>> >>> It seems also that images don't have any 'cache' attributes. >>> (I don't recommended using ETags unless you're absolutely sure that >>> ETags will be always the >>> same on multiple web servers. i.e. some webservers use inode >>> information to compute the ETag >>> which is different on different servers...thought I don't know how >>> erlang-web compute it) >> >> Please inform a newbie, I just checked HTML4.01 and did not >> find any "cache" attribute. > > He's referring to cache control headers such as Expires, ETag, and > Cache-Control. These aren't part of HTML but rather are typically > configured for the web server itself. See section 13 of the HTTP 1.1 > spec: > > http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13 Exactly Steve, Raimo, Here's a simple method to determine how 'caching' is handled by the webserver: http://www.ircache.net/cgi-bin/cacheability.py?query=http%3A%2F%2Fwww.erlang.org%2Fimages%2Fsearch_bar.jpg&descend=on It seems that inets isn't really ideal for serving real web content, because [extracted from the web page]: Last-Modified 70 weeks 5 days ago (Fri, 20 Nov 2009 11:37:29 GMT) validation returned same object If-Modified-Since are HTTP headers means: me (web browser talking) please web server give me only content that has changed since this "date", because I don't need to retrieve the same content since I already have it... Unfortunately inets sends the same content every time a HTTP GET is done, regardless of HTTP headers. I hope that this explanation will clarify a bit more. From norton@REDACTED Thu Mar 31 02:43:14 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Thu, 31 Mar 2011 09:43:14 +0900 Subject: [erlang-questions] Re: Programatically Starting Erlang Nodes In-Reply-To: <AANLkTinMr+Ce7dJUrs-6V_29mL64OiRfWqhZrFaJ4GBa@mail.gmail.com> References: <AANLkTinMr+Ce7dJUrs-6V_29mL64OiRfWqhZrFaJ4GBa@mail.gmail.com> Message-ID: <op.vs6mybd0raxtje@norton-pc.geminimobile.com> Dale - There is a recipe described in this README (https://github.com/hibari/clus) for setting up a cluster of Hibari nodes. The recipe isn't necessarily elegant but it is simple and can be customized as needed. The script clus.sh is used for creating and for setting up new UNIX accounts. It is non specific to Hibari. The script clus-hibari.sh is used for Hibari specific operations. The script could be copied and customized to your own needs. thanks, Joe N -- norton@REDACTED From erlang@REDACTED Thu Mar 31 09:04:29 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 31 Mar 2011 09:04:29 +0200 Subject: [erlang-questions] Re: about a tcp question in erlang In-Reply-To: <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> Message-ID: <AANLkTi=H6kOZM6ObDMzFN2_6Y10CG=qr4eiaLfPtxvwP@mail.gmail.com> On Tue, Mar 29, 2011 at 5:45 AM, zhangbo <bo.zhang86@REDACTED> wrote: > Hi, > Code: > {ok, Listen} = gen_tcp:listen(80, [{packet,0}, > binary, > {active, true}]) > > Loop: > receive > {tcp, S, Bin} -> > ... > > I want to know, when a client send a message("helloworld") to server, then > send another message("helloworld2"). How many messages will the server > receive? (I guess 2.) If the client send very fast, or the message is big > enough, how the situation again? > > When receive {tcp, S, Bin}, Bin stands for a message or maybe just a > fragment of message? If the client send "hello, server", Bin maybe "hello, > server", but If the message is very big, what will Bin be? and how many > times will the loop receive? > because you have said {packet,0} then data might become fragmented. The options {packet,2}, (or {packet,4} mean "send a 2(4) byte length header at the start of the data". If the client and server both open a socket with {packet,4} then any fragmentation will be taken care of by the tcp driver. In this case any packet fragments will be recombined by the driver and only completed messages delivered to the application. /Joe > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/a2b5e527/attachment.htm> From yuchao1@REDACTED Thu Mar 31 09:15:55 2011 From: yuchao1@REDACTED (=?UTF-8?B?5L2Z6LaF?=) Date: Thu, 31 Mar 2011 15:15:55 +0800 Subject: [erlang-questions] Re: about a tcp question in erlang In-Reply-To: <AANLkTi=H6kOZM6ObDMzFN2_6Y10CG=qr4eiaLfPtxvwP@mail.gmail.com> References: <87273C4B-2BD8-479C-81A2-E2091479757F@gmail.com> <FBD3A46D-E594-46D6-BE6D-0C6ED31BCF6E@gmail.com> <AANLkTi=H6kOZM6ObDMzFN2_6Y10CG=qr4eiaLfPtxvwP@mail.gmail.com> Message-ID: <4D942A2B.2010003@staff.sina.com.cn> ? 2011/3/31 15:04, Joe Armstrong ??: > > > On Tue, Mar 29, 2011 at 5:45 AM, zhangbo <bo.zhang86@REDACTED > <mailto:bo.zhang86@REDACTED>> wrote: > > Hi, > Code: > {ok, Listen} = gen_tcp:listen(80, [{packet,0}, > binary, > {active, true}]) > > Loop: > receive > {tcp, S, Bin} -> > ... > > I want to know, when a client send a message("helloworld") to > server, then send another message("helloworld2"). How many > messages will the server receive? (I guess 2.) If the client send > very fast, or the message is big enough, how the situation again? > > When receive {tcp, S, Bin}, Bin stands for a message or maybe just > a fragment of message? If the client send "hello, server", Bin > maybe "hello, server", but If the message is very big, what will > Bin be? and how many times will the loop receive? > > > because you have said {packet,0} then data might become fragmented. > The options {packet,2}, (or {packet,4} mean "send a 2(4) byte length > header at the start of the data". If the client and server > both open a socket with {packet,4} then any fragmentation will be > taken care of by the tcp driver. In this > case any packet fragments will be recombined by the driver and only > completed messages delivered to > the application. > > /Joe > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED <mailto:erlang-questions@REDACTED> > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joe? nice? thanks -- Best Regards ?? YuChao UC:yuchao86@REDACTED ????? ?????? ????8610?62676661 ???13466539920 MSN:yuchao86@REDACTED ??????????????58???????15?22-2 ___________________________________________ http://www.sina.com.cn You're the one ??.?? ?????? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/50ee764f/attachment.htm> From raimo+erlang-questions@REDACTED Thu Mar 31 09:25:09 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 31 Mar 2011 09:25:09 +0200 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <A6F075E2-36F4-450C-9118-34C9C5D700CB@gmail.com> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> <AANLkTikZoHLuUL4y1WJ7S91eWFp5-iiu2-X8oYU9tAuO@mail.gmail.com> <A6F075E2-36F4-450C-9118-34C9C5D700CB@gmail.com> Message-ID: <20110331072509.GA12516@erix.ericsson.se> On Thu, Mar 31, 2011 at 12:07:55AM +0200, Antoine Koener wrote: > > On Mar 30, 2011, at 13:50 , Steve Vinoski wrote: > > >On Wed, Mar 30, 2011 at 7:41 AM, Raimo Niskanen > ><raimo+erlang-questions@REDACTED> wrote: > >>On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: > >>>Cacheability ? > >>> > >>>I can reload the page multiples times and still download images. > >>> > >>>It seems also that images don't have any 'cache' attributes. > >>>(I don't recommended using ETags unless you're absolutely sure that > >>>ETags will be always the > >>>same on multiple web servers. i.e. some webservers use inode > >>>information to compute the ETag > >>>which is different on different servers...thought I don't know how > >>>erlang-web compute it) > >> > >>Please inform a newbie, I just checked HTML4.01 and did not > >>find any "cache" attribute. > > > >He's referring to cache control headers such as Expires, ETag, and > >Cache-Control. These aren't part of HTML but rather are typically > >configured for the web server itself. See section 13 of the HTTP 1.1 > >spec: > > > >http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13 > > Exactly Steve, > > Raimo, > > Here's a simple method to determine how 'caching' is handled by the > webserver: > http://www.ircache.net/cgi-bin/cacheability.py?query=http%3A%2F%2Fwww.erlang.org%2Fimages%2Fsearch_bar.jpg&descend=on > > It seems that inets isn't really ideal for serving real web content, > because [extracted from the web page]: > Last-Modified 70 weeks 5 days ago (Fri, 20 Nov 2009 11:37:29 GMT) > validation returned same object > > If-Modified-Since are HTTP headers means: > me (web browser talking) please web server give me only content that > has changed since this "date", because I don't need to > retrieve the same content since I already have it... > > Unfortunately inets sends the same content every time a HTTP GET is > done, regardless of HTTP headers. > > I hope that this explanation will clarify a bit more. It seems so, thank you very much. Now I have something to dig into. Part of the reason why we run Inets on the site is to find out what problems it has. We supposedly can change web server to Yaws or even maybe Mochiweb (I do not remember), but would preferably take the opportunity to fix Inets. I am not on the Inets team so I will take this to them... > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlangsiri@REDACTED Thu Mar 31 10:16:43 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 31 Mar 2011 10:16:43 +0200 Subject: [erlang-questions] Re: supervisors In-Reply-To: <AANLkTik8_FmG1hipmUkmD2oKgS36XPKsee-P52Hsgz2q@mail.gmail.com> References: <AANLkTik8_FmG1hipmUkmD2oKgS36XPKsee-P52Hsgz2q@mail.gmail.com> Message-ID: <AANLkTimpGhNCFV0b0XQAmgcKSfijE5iL0T20qL5HerQO@mail.gmail.com> Hi Wes, It should be possible to do exactly what you want. I have tried to recreate your problem, but I did not succeed. If you want to you may post the code for the two supervisors and I will have a look at it to see what I'm doing different. Regards /siri 2011/3/30 Wes James <comptekki@REDACTED> > when I start and app that works fine: > > =PROGRESS REPORT==== 30-Mar-2011::11:28:05 === > supervisor: {local,eapp_sup} > started: [{pid,<0.70.0>}, > {name,{a_sup,a1}}, > {mfargs,{a_sup,start_link,[a1]}}, > {restart_type,permanent}, > {shutdown,brutal_kill}, > {child_type,supervisor}] > > In eapp_sup I have: > > start_a(Module, Name) -> > supervisor:start_child(?MODULE, child(Module, Name)). > > > child(Module, Name) -> > {{Module, Name}, {Module, start_link, [Name]}, permanent, > brutal_kill, supervisor, [Name]}. > > > from starting above it would seem that a_sup has the name a1, but if > you run appmon it shows just as a_sup. > > When I then run > > eapp:start_a(a_sup,a2). I would assume that a_sup could load as a > second supervisor process with name a2, but this is what I get: > > eapp_sup:start_a(a_sup,a2). > {error,{already_started,<0.70.0>}} > > I then create a_sup2 and eapp_sup:start_a(a_sup2, a1) and that loads > fine, since the module "name" is not the same in the system as a_sup. > But it seems like a name should be assigned to these to make them > unique by name and be able to use the same supervisor code to > supervise some more children. > > Is that possible? > > thx, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/454eed71/attachment.htm> From mytake6@REDACTED Thu Mar 31 11:13:19 2011 From: mytake6@REDACTED (lei li) Date: Thu, 31 Mar 2011 17:13:19 +0800 Subject: [erlang-questions] Re: [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend In-Reply-To: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> References: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> Message-ID: <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> Hi Jack, Congratulation! Can you open source your erlang code? On Thu, Mar 31, 2011 at 3:37 AM, Jack Moffitt <jack@REDACTED> wrote: > Hi all, > > I just wanted to tell everyone that my iOS game, Snack Words, finally > launched a week or so ago. I'm a pretty big fan of word games, and I > was really surprised there are so few multiplayer games in the App > Store. > > It seemed like a perfect job for Erlang too, and I gave a talk about > it at recent Erlang Factory in San Francisco. The full server > implementation, including rating system, pairing, and robot players > clocks in at ~2,400 lines of code, including comments and whitespace. > There's another 900 lines of tests. > > It really amazed me at the end how reading the main client gen_fsm > code is basically just a funnily formated description of the protocol > itself. > > Perhaps I'll be able to write the client side of future games in Erlang soon :) > > Hope you enjoy! > > jack. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Thu Mar 31 11:13:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 31 Mar 2011 10:13:23 +0100 Subject: [erlang-questions] Re: linked-in driver In-Reply-To: <4D9366BF.6080106@bobcowdery.plus.com> References: <4D9366BF.6080106@bobcowdery.plus.com> Message-ID: <AANLkTinqYpLvWt_hKGjf_uj7c1vkadBJeYhtwmJU7DqX@mail.gmail.com> > When I run it I get arity of 0 when it should be 2. > > What am I doing wrong? Did you remember to ei_decode_version first? From alexey.v.romanov@REDACTED Thu Mar 31 11:54:19 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Thu, 31 Mar 2011 13:54:19 +0400 Subject: [erlang-questions] Do gen_tcp sockets support recv_timeout option? Message-ID: <AANLkTi=XCG4JGjhbBzwHFB=4afbU=YpAikD00ikN3FnR@mail.gmail.com> It isn't listed in http://www.erlang.org/doc/man/inet.html#setopts-2, but Googling finds some uses: e.g. https://bitbucket.org/basho/casbench/src/7bba90cbf3e9/src/thrift_socket_server.erl Yours, Alexey Romanov From emile@REDACTED Thu Mar 31 14:14:39 2011 From: emile@REDACTED (Emile Joubert) Date: Thu, 31 Mar 2011 13:14:39 +0100 Subject: [erlang-questions] Re: crypto on 64bit windows? In-Reply-To: <AANLkTi=DcfchKRp0ncQujUNWs0Lh8gBYdQsv3YFFhhWM@mail.gmail.com> References: <4D91B095.3040600@rabbitmq.com> <AANLkTikNvGDBaZtjObaUV_jiy2Nbey5Y-fJOyK4KbEYB@mail.gmail.com> <AANLkTi=DcfchKRp0ncQujUNWs0Lh8gBYdQsv3YFFhhWM@mail.gmail.com> Message-ID: <4D94702F.8060509@rabbitmq.com> On 29/03/11 18:10, Robert Raschke wrote: > The crypto.dll needs to find libeay32.dll (on 32-bit systems at > least, not sure what it is called on 64-bit) I can confirm that crypto works if you install the 32bit libraries on a 64bit system. Thanks for the help. -Emile From koops.j@REDACTED Thu Mar 31 14:36:33 2011 From: koops.j@REDACTED (Jeroen Koops) Date: Thu, 31 Mar 2011 14:36:33 +0200 Subject: [erlang-questions] Re: Distributed application takeover In-Reply-To: <0DBAD7AB-FE6B-4A54-A07B-C8ABAB6E8ACF@erlang-solutions.com> References: <AANLkTi=RuSSaeR_w4TLFJeKW6-0DhfQ5a78RCZ1Uq668@mail.gmail.com> <0DBAD7AB-FE6B-4A54-A07B-C8ABAB6E8ACF@erlang-solutions.com> Message-ID: <AANLkTikGaU2B5B4Udk7gyJGgekMPEGPD5DdOkLEajoRO@mail.gmail.com> Hi Ulf, Thanks for the explanation, the message-flow is now pretty clear. Actually, the reason I was looking into it was to see if I could come up with a patch that makes it possible to configure an alternative takeover behaviour, since I find it inconvenient that during a takeover the application is running on two nodes at the same time for a while, which leads to clashes with globally registered names, and problems with processes that assume they have exclusive access to data. It would be nice to be able to configure takeover behaviour in such a way that the application is only started on the node that is taking over *after *it has been stopped on the original node. However, I've come to the conclusion that the whole application distribution is way too complex to start poking around, so instead I now use a mechanism where, whenever an application is started with { takeover, OtherNode } as StartType, it simply terminates all the children of the application's top-level supervisor on OtherNode before proceeding with the startup. This is a bit clunky, and a feel a bit sorry for the application on OtherNode being left behind as an empty shell, with all its children killed, waiting for the application controller to put it out of its misery, but it gets the job done... On Wed, Mar 30, 2011 at 4:00 PM, Ulf Wiger <ulf.wiger@REDACTED>wrote: > > Hi Jeroen, > > This is a bit tricky, and can be very confusing for someone trying to build > their own cluster controller. :) > > The stopping of the application is done by the local application_controller > when it is told that the application is running somewhere else. In other > words, dist_ac kindly forwards the information to the local AC, and it is > the local AC that in this particular case takes responsibility for 'knowing' > that the local instance should be stopped. > > See application_controller:handle_application_started/3 > > > https://github.com/erlang/otp/blob/OTP_R14B02/lib/kernel/src/application_controller.erl#L913 > > (The dirty deed actually happens on line 946). > > The reason I know this, is that in the AXD 301, we ran multiple instances > of some applications, distributed across several mated pairs - each instance > having its own standby node. This can be done by writing an own distributed > AC, but it has to be smart enough to know _when_ to forward the {started, > Node} status to the local AC; if an instance was running locally - and _was > supposed to_ do so (i.e. not involved in a takeover), the distributed AC had > to suppress this message. > > BR, > Ulf W > > On 30 Mar 2011, at 14:11, Jeroen Koops wrote: > > Hi, > > I'm trying to find out how the distributed application controller works > internally. I'm especially interested in the implementation of an > application takeover. > > In case an application runs on node A, and is taken over by node B, what > should happen is that it is first started on node B, so that there are two > instances of the application running simultaneously for a brief period of > time, and then stopped on node A. > > However, I cannot figure out where this stopping happens in dist_ac.erl. > If I understand correctly, this should happen in response to a > ac_application_run message from the application_controller. This message > is received by the dist_ac on node B, and a dist_ac_app_started message is > then broadcast to the dist_acs on all connected nodes. The dist_ac of node > A receives this message, notices that the application is still running > locally, and decides to shut down the application on its own node -- at > least that is what the comments say (dist_ac.erl, line 529): > > %% Another node tookover from me; stop my application > %% and update the running list. > > But all I can see is that the dist_ac's list of applications is updated to > indicate that the application is no longer running locally -- I cannot find > where the application_controller is instructed to actually shutdown the > application. > > Can anyone point me in the right direction? > > Thanks, > > Jeroen _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/b73f5785/attachment.htm> From vinoski@REDACTED Thu Mar 31 16:15:29 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 31 Mar 2011 10:15:29 -0400 Subject: [erlang-questions] Re: [erlang-questions 29] Re: fun with the new erlang.org! In-Reply-To: <20110331072509.GA12516@erix.ericsson.se> References: <20110325185008.GA32567@ecn.lan> <4D8D799E.7020603@eonblast.com> <87527205-E4AB-482D-8436-D01EFD809676@gmail.com> <20110330114139.GA28211@erix.ericsson.se> <AANLkTikZoHLuUL4y1WJ7S91eWFp5-iiu2-X8oYU9tAuO@mail.gmail.com> <A6F075E2-36F4-450C-9118-34C9C5D700CB@gmail.com> <20110331072509.GA12516@erix.ericsson.se> Message-ID: <AANLkTikPdzOr9Mh_kbzmbT1_XqaLHMdAT1NB8_+5VWdF@mail.gmail.com> On Thu, Mar 31, 2011 at 3:25 AM, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote: > On Thu, Mar 31, 2011 at 12:07:55AM +0200, Antoine Koener wrote: >> >> On Mar 30, 2011, at 13:50 , Steve Vinoski wrote: >> >> >On Wed, Mar 30, 2011 at 7:41 AM, Raimo Niskanen >> ><raimo+erlang-questions@REDACTED> wrote: >> >>On Sat, Mar 26, 2011 at 05:03:56PM +0100, Koener Antoine wrote: >> >>>Cacheability ? >> >>> >> >>>I can reload the page multiples times and still download images. >> >>> >> >>>It seems also that images don't have any 'cache' attributes. >> >>>(I don't recommended using ETags unless you're absolutely sure that >> >>>ETags will be always the >> >>>same on multiple web servers. i.e. some webservers use inode >> >>>information to compute the ETag >> >>>which is different on different servers...thought I don't know how >> >>>erlang-web compute it) >> >> >> >>Please inform a newbie, I just checked HTML4.01 and did not >> >>find any "cache" attribute. >> > >> >He's referring to cache control headers such as Expires, ETag, and >> >Cache-Control. These aren't part of HTML but rather are typically >> >configured for the web server itself. See section 13 of the HTTP 1.1 >> >spec: >> > >> >http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13 >> >> Exactly Steve, >> >> Raimo, >> >> Here's a simple method to determine how 'caching' is handled by the >> webserver: >> http://www.ircache.net/cgi-bin/cacheability.py?query=http%3A%2F%2Fwww.erlang.org%2Fimages%2Fsearch_bar.jpg&descend=on >> >> It seems that inets isn't really ideal for serving real web content, >> because [extracted from the web page]: >> Last-Modified ?70 weeks 5 days ago ?(Fri, 20 Nov 2009 11:37:29 GMT) >> validation returned same object >> >> If-Modified-Since are HTTP headers means: >> ?me (web browser talking) please web server give me only content that >> has changed since this "date", because I don't need to >> retrieve the same content since I already have it... >> >> Unfortunately inets sends the same content every time a HTTP GET is >> done, regardless of HTTP headers. >> >> I hope that this explanation will clarify a bit more. > > It seems so, thank you very much. Now I have something to dig into. > Part of the reason why we run Inets on the site is to find out what > problems it has. We supposedly can change web server to Yaws or even > maybe Mochiweb (I do not remember), but would preferably take the > opportunity to fix Inets. I am not on the Inets team so I will take > this to them... Also a bit unusual is the website's failure to respond to HEAD requests, as this verbose output from curl shows: $ curl -Iv http://www.erlang.org/ * About to connect() to www.erlang.org port 80 (#0) * Trying 192.121.151.107... connected * Connected to www.erlang.org (192.121.151.107) port 80 (#0) > HEAD / HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: www.erlang.org > Accept: */* > * Empty reply from server * Connection #0 to host www.erlang.org left intact curl: (52) Empty reply from server * Closing connection #0 A website technically doesn't need to respond to HEAD requests but for purposes of cache coherency it's common for caches to use both conditional GETs (which is why supporting ETag and cache control headers is important) and HEAD requests for checking for updates on origin servers. --steve From michael.santos@REDACTED Thu Mar 31 17:05:41 2011 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 31 Mar 2011 11:05:41 -0400 Subject: [erlang-questions] Re: Do gen_tcp sockets support recv_timeout option? In-Reply-To: <AANLkTi=XCG4JGjhbBzwHFB=4afbU=YpAikD00ikN3FnR@mail.gmail.com> References: <AANLkTi=XCG4JGjhbBzwHFB=4afbU=YpAikD00ikN3FnR@mail.gmail.com> Message-ID: <20110331150541.GA11652@ecn.lan> On Thu, Mar 31, 2011 at 01:54:19PM +0400, Alexey Romanov wrote: > It isn't listed in http://www.erlang.org/doc/man/inet.html#setopts-2, > but Googling finds some uses: > > e.g. https://bitbucket.org/basho/casbench/src/7bba90cbf3e9/src/thrift_socket_server.erl This code is using recv_timeout as the timeout parameter for a gen_tcp:recv/3 ({active,false} socket). See: https://bitbucket.org/basho/casbench/src/4265a37d2200/src/thrift_socket_transport.erl#cl-51 From gnoblin@REDACTED Thu Mar 31 17:37:18 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Thu, 31 Mar 2011 18:37:18 +0300 Subject: [erlang-questions] Re: [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend In-Reply-To: <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> References: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> Message-ID: <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> Nice! Could you also post iTunes link here? thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/8c799175/attachment.htm> From comptekki@REDACTED Thu Mar 31 17:38:38 2011 From: comptekki@REDACTED (Wes James) Date: Thu, 31 Mar 2011 09:38:38 -0600 Subject: [erlang-questions] Re: supervisors In-Reply-To: <AANLkTimpGhNCFV0b0XQAmgcKSfijE5iL0T20qL5HerQO@mail.gmail.com> References: <AANLkTik8_FmG1hipmUkmD2oKgS36XPKsee-P52Hsgz2q@mail.gmail.com> <AANLkTimpGhNCFV0b0XQAmgcKSfijE5iL0T20qL5HerQO@mail.gmail.com> Message-ID: <AANLkTikx96jxaUYz8QTt=9pUDo0Z8cNfGOWcyDxFDhNu@mail.gmail.com> Siri, The code is from last weeks training part of the Erlang Factory in San Francisco. I don't know if Erlang Solutions would like me sending all the code to people or not. I'd send it to you off-list if it was ok with them. Or maybe you could send me a sample of your code to compare with what I have. thx, -wes On Thu, Mar 31, 2011 at 2:16 AM, Siri Hansen <erlangsiri@REDACTED> wrote: > Hi Wes, > It should be possible to do exactly what you want.?I have tried to recreate > your problem, but I did not succeed. If you want to you may post the code > for the two supervisors and I will have a look at it to see what I'm doing > different. > Regards > /siri > > > 2011/3/30 Wes James <comptekki@REDACTED> >> >> when I start and app that works fine: >> >> =PROGRESS REPORT==== 30-Mar-2011::11:28:05 === >> ? ? ? ? ?supervisor: {local,eapp_sup} >> ? ? ? ? ? ? started: [{pid,<0.70.0>}, >> ? ? ? ? ? ? ? ? ? ? ? {name,{a_sup,a1}}, >> ? ? ? ? ? ? ? ? ? ? ? {mfargs,{a_sup,start_link,[a1]}}, >> ? ? ? ? ? ? ? ? ? ? ? {restart_type,permanent}, >> ? ? ? ? ? ? ? ? ? ? ? {shutdown,brutal_kill}, >> ? ? ? ? ? ? ? ? ? ? ? {child_type,supervisor}] >> >> In eapp_sup I have: >> >> start_a(Module, Name) -> >> ? ? ? ?supervisor:start_child(?MODULE, child(Module, Name)). >> >> >> child(Module, Name) -> >> ?{{Module, Name}, {Module, start_link, [Name]}, permanent, >> brutal_kill, supervisor, [Name]}. >> >> >> from starting above it would seem that a_sup has the name a1, but if >> you run appmon it shows just as a_sup. >> >> When I then run >> >> eapp:start_a(a_sup,a2). ?I would assume that a_sup could load as a >> second supervisor process with name a2, but this is what I get: >> >> eapp_sup:start_a(a_sup,a2). >> {error,{already_started,<0.70.0>}} >> >> I then create a_sup2 ?and eapp_sup:start_a(a_sup2, a1) and that loads >> fine, since the module "name" is not the same in the system as a_sup. >> But it seems like a name should be assigned to these to make them >> unique by name and be able to use the same supervisor code to >> supervise some more children. >> >> Is that possible? >> >> thx, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From s.j.thompson@REDACTED Thu Mar 31 17:48:53 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Thu, 31 Mar 2011 08:48:53 -0700 Subject: [erlang-questions] Trends in Functional Programming: last CFP Message-ID: <1C29691D-EC0E-4D9C-88A6-0184C6E15AB8@kent.ac.uk> [Apologies for any duplication, Simon] ********************************************************* LAST CALL FOR PAPERS and CALL FOR PARTICIPATION 12th International Symposium Trends in Functional Programming 2011 Madrid, Spain May 16-18, 2011 http://dalila.sip.ucm.es/tfp11 ********************************************************* SUBMISSION AND REGISTRATION DATES APPROACHING (2011) Full papers/extended abstracts submission: April 2nd Notification of acceptance for presentation: April 15th Early registration deadline: April 25th Camera ready for draft proceeding: April 30th INVITED SPEAKER In this TFP edition, an invited talk will be given by Neil Mitchell (http://community.haskell.org/~ndm/), who finished his PhD thesis on 'Transformation and Analysis of Functional Programs' at the University of York, England, and is currently working for the Standard Chartered Bank. The title of the talk is 'Finding functions from types', and will be about the Hoogle tool (http://haskell.org/hoogle). DESCRIPTION The symposium on Trends in Functional Programming (TFP) is an international forum for researchers with interests in all aspects of functional programming, taking a broad view of current and future trends in the area. It aspires to be a lively environment for presenting the latest research results, and other contributions (see below), described in draft papers submitted prior to the symposium. A formal post-symposium refereeing process then selects a subset of the articles presented at the symposium and submitted for formal publication, as a Springer Lecture Notes in Computer Science volume, as they were for the TFP-2010 selected papers. TFP 2011 is going to be held in the Computer Science Faculty of Complutense University of Madrid, on May 16-18, 2011. It will be co-located with the 2nd International Workshop on Foundational and Practical Aspects of Resource Analysis (FOPARA 2011) (http://dalila.sip.ucm.es/fopara11/). This collocation could make such a gathering a very interesting event and will allow researchers from the two communities to exchange ideas. The TFP symposium is the heir of the successful series of Scottish Functional Programming Workshops. Previous TFP symposia were held in Edinburgh (Scotland) in 2003, in Munich (Germany) in 2004, in Tallinn (Estonia) in 2005, in Nottingham (UK) in 2006, in New York (USA) in 2007, in Nijmegen (The Netherlands) in 2008, in Komarno (Slovakia) in 2009, and in Oklahoma (USA) in 2010. For further general information about TFP please see the TFP homepage at http://www.tifp.org/. SCOPE OF THE SYMPOSIUM The symposium recognises that new trends may arise through various routes. As part of the Symposium's focus on trends we therefore identify the following five article categories. High-quality articles are solicited in any of these categories: Research Articles: leading-edge, previously unpublished research work Position Articles: on what new trends should or should not be Project Articles: descriptions of recently started new projects Evaluation Articles: what lessons can be drawn from a finished project Overview Articles: summarising work with respect to a trendy subject Articles must be original and not submitted for simultaneous publication to any other forum. They may consider any aspect of functional programming: theoretical, implementation-oriented, or more experience oriented. Applications of functional programming techniques to other languages are also within the scope of the symposium. Articles on the following subject areas are particularly welcome: o Dependently typed functional programming o Validation and verification of functional programs o Debugging for functional languages o Functional programming in different application areas: security, mobility, telecommunications applications, embedded systems, global computing, grids etc. o Functional languages for reasoning about imperative/object-oriented programs o Interoperability with imperative programming languages o Novel memory management techniques o Program transformation techniques o Empirical performance studies o Abstract/virtual machines and compilers for functional languages o New implementation strategies o Any new emerging trend in the functional programming area If you are in doubt on whether your article is within the scope of TFP, please contact the TFP 2011 program chair, Ricardo Pe~na, at tfp2011@REDACTED BEST STUDENT PAPER AWARD TFP traditionally pays special attention to research students, acknowledging that students are almost by definition part of new subject trends. A student paper is one for which the authors state that the paper is mainly the work of students, the students are listed as first authors, and a student would present the paper. A prize for the best student paper is awarded each year. SUBMISSION AND DRAFT PROCEEDINGS Acceptance of articles for presentation at the symposium is based on a lightweight peer review process of extended abstracts (6 to 10 pages in length) or full papers (16 pages). Accepted abstracts are to be completed to full papers before the symposium for publication in the draft proceedings. The submission must clearly indicate which category it belongs to: research, position, project, evaluation, or overview paper. It should also indicate whether the main author or authors are research students. Formatting details can be found at the TFP 2011 website. Submission procedures will be posted on the TFP 2011 website as the submission deadline is approaching. The papers of the local proceedings will also be made available on-line under some copyright conditions, with which all authors are asked to agree (see http://dalila.sip.ucm.es/tfp11/). POST-SYMPOSIUM REFEREEING AND PUBLICATION In addition to the symposium draft proceedings, we will continue the last year decision of publishing a high-quality subset of contributions in the Springer Lecture Notes in Computer Science series (previous editions were published by Intellect). All TFP authors will be invited to submit revised papers after the symposium. These will be refereed using normal conference standards and a subset of the submitted papers, over all categories, will be selected for publication. Papers will be judged on their contribution to the research area with appropriate criteria applied to each category of paper. Student papers will be given extra feedback by the Program Committee in order to assist those unfamiliar with the publication process. Important dates (2011): TFP 2011 Symposium: May 16-18th Student papers feedback: June 6th Submission for formal review: June 24th Notification of acceptance for LNCS: September 2nd Camera ready paper: September 23rd TFP 2011 ORGANIZATION Steering Committee Chair: Marko van Eekelen, Radboud University Nijmegen and Open University, NL Steering Committee Treasurer: Hans-Wolfgang Loidl, Heriot-Watt University, UK Symposium Organization Chair: Ricardo Pe~na, Complutense University of Madrid, ES TFP 2011 PROGRAMME COMMITTEE Peter Achten (Radboud University Nijmegen, NL) Ana Bove (Chalmers University of Technology, SE) Olaf Chitil (University of Kent, UK) Marko van Eekelen (Radboud University Nijmegen,Open University, NL) Robby Findler (Northwestern University, USA) Victor Gul'ias (University of La Coru~na, ES) Jurriaan Hage (University of Utrecht, NL) Kevin Hammond (University of St. Andrews, UK) Michael Hanus (Christian Albrechts University zu Kiel, DE) Zolt'an Horv'ath (E"otv"os Lor'and University, HU) Frank Huch (Christian Albrechts University zu Kiel, DE) Mauro Jaskelioff (National University of Rosario, AR) Rita Loogen (Philipps University Marburg, DE) Jay McCarthy (Brigham Young University, USA) Henrik Nilsson (University of Nottingham, UK) Rex Page (University of Oklahoma, USA) Ricardo Pe~na (Chair) (Complutense University of Madrid, ES) John Reppy (University of Chicago, USA) Konstantinos Sagonas (Uppsala University, SE, and National Technical University of Athens, GR) Simon Thompson (University of Kent, UK) German Vidal (Universidad Polit'ecnica de Valencia, ES) SPONSORS Computer Science Faculty, Complutense University of Madrid Spanish Ministry of Science and Innovation From richardprideauxevans@REDACTED Thu Mar 31 17:49:54 2011 From: richardprideauxevans@REDACTED (Richard Evans) Date: Thu, 31 Mar 2011 16:49:54 +0100 Subject: [erlang-questions] Re: [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend In-Reply-To: <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> References: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> Message-ID: <AANLkTimSnFWK5BVKSmm9FtOyML3GAFw0E6et3qvqfr9M@mail.gmail.com> http://itunes.apple.com/gb/app/snack-words/id425786167?mt=8 I downloaded this today; it is really well executed and good fun. The bite-sized chunks of gameplay work very well on the iphone. Good job sir. On Thu, Mar 31, 2011 at 4:37 PM, Slav Pankratov <gnoblin@REDACTED> wrote: > Nice! > Could you also post iTunes link here? > > thanks, > Slav > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/cade4f3b/attachment.htm> From pablo.platt@REDACTED Thu Mar 31 18:02:42 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Thu, 31 Mar 2011 09:02:42 -0700 (PDT) Subject: [erlang-questions] Re: about a tcp question in erlang Message-ID: <230025.18826.qm@web112612.mail.gq1.yahoo.com> The {packet,2}, (or {packet,4} is great in cases that it can be used. For example, in the mongodb wire protocol the size header is int32 little-endian which include the int32 (packet size+4Bytes). http://www.mongodb.org/display/DOCS/Mongo+Wire+Protocol#MongoWireProtocol-MessagesTypesandFormats Is there a chance this feature be extended to support little/big endianess and offset? It was previously requested here: http://erlang.2086793.n4.nabble.com/packet-PacketType-for-little-signed-packets-td2244372.html ________________________________ From: Joe Armstrong <erlang@REDACTED> To: zhangbo <bo.zhang86@REDACTED> Cc: erlang-questions@REDACTED Sent: Thursday, March 31, 2011 9:04 AM Subject: [erlang-questions] Re: about a tcp question in erlang On Tue, Mar 29, 2011 at 5:45 AM, zhangbo <bo.zhang86@REDACTED> wrote: Hi, >? ? ? ?Code: >? ? ? ?{ok, Listen} = gen_tcp:listen(80, [{packet,0}, >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?binary, >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{active, true}]) > >? ? ? ?Loop: >? ? ? ? ? ? ? ?receive >? ? ? ? ? ? ? ? ? ? ? ?{tcp, S, Bin} -> >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?... > >I want to know, when a client send a message("helloworld") to server, then send another message("helloworld2"). How many messages will the server receive? (I guess 2.) ?If the client send very fast, or the message is big enough, how the situation again? > >When receive {tcp, S, Bin}, Bin stands for a message or maybe just a fragment of message? If the client send "hello, server", Bin maybe "hello, server", but If the message is very big, what will Bin be? and how many times will the loop receive? > because you have said {packet,0} then data might become fragmented. The options {packet,2}, (or {packet,4} mean "send a 2(4) byte length header at the start of the data". If the client and server both open a socket with {packet,4} then any fragmentation will be taken care of by the tcp driver. In this case any packet fragments will be recombined by the driver and only completed messages delivered to the application. /Joe ? ? > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/d544eef0/attachment.htm> From ulf.wiger@REDACTED Thu Mar 31 18:20:35 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 31 Mar 2011 18:20:35 +0200 Subject: [erlang-questions] how to add a module to preLoaded? Message-ID: <BEC404E6-42CE-4C6F-BC0E-00A93D0E8AFE@erlang-solutions.com> I'm trying to add a preloaded module for a patch I'm working on, but this turns out to be exceptionally difficult. I found the ./otp_build update_preloaded, which works great for existing preloaded files. Now trying to add (in my case) prim_ram_file, to the list of preloaded modules, I've so far found the list hard-coded in the following places: - erts/preloaded/src/Makefile - $ERL_TOP/Makefile.in - lib/sasl/src/systools_make.erl I feel that I have updated all the hard-coded representations, and tried various combinations of - ./otp_build update_preloaded - make - make release - ./otp_build setup - ./otp_build boot - ./otp_build update_primary None of them seem to actually change the start scripts - not that I'm even sure that this would make a difference, since the whole preloading is a deep dark mystery. What is the magic incantation that I'm missing? BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jack@REDACTED Thu Mar 31 18:31:22 2011 From: jack@REDACTED (Jack Moffitt) Date: Thu, 31 Mar 2011 10:31:22 -0600 Subject: [erlang-questions] Re: [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend In-Reply-To: <AANLkTimSnFWK5BVKSmm9FtOyML3GAFw0E6et3qvqfr9M@mail.gmail.com> References: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> <AANLkTimSnFWK5BVKSmm9FtOyML3GAFw0E6et3qvqfr9M@mail.gmail.com> Message-ID: <AANLkTini5kBA3DpM4RTGq8Cis5o6wj9ggUp5BwB8-xwr@mail.gmail.com> > I downloaded this today; it is really well executed and good fun. The > bite-sized chunks of gameplay work very well on the iphone. > Good job sir. Thanks for the kind words! As for open sourcing the server, I'm not sure I'll do that, although you never know. I did show quite a lot of the structure and some of the code during my talk, which I assume will be up as a video soon. jack. From jared.nance@REDACTED Thu Mar 31 19:30:30 2011 From: jared.nance@REDACTED (Jared Kofron) Date: Thu, 31 Mar 2011 10:30:30 -0700 Subject: [erlang-questions] Re: [ANN] Snack Words - Real-time, multi-player iOS game with Erlang backend In-Reply-To: <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> References: <AANLkTikD49YhiUep+Yt2B9xtDF9KJ02SPbh9BqvgPPt9@mail.gmail.com> <AANLkTinYdjuPW9yB8MudTt5WpsaLykeFcqtUP1LTJD1Z@mail.gmail.com> <AANLkTikNekGKX2Nx_fb=mJYyO1SC1yJ6z-GZ4-Of_FH5@mail.gmail.com> Message-ID: <FBD8BFAE-F8DC-458D-996D-0F6572DD25A6@gmail.com> I actually also recently started with an Erlang backend for an iOS game, but ended up using SimpleDB (technically also an erlang backend) because of hosting issues. Did you wind up using a hosting service? If so, can you recommend it for hosted erlang? On Mar 31, 2011, at 8:37 AM, Slav Pankratov wrote: > Nice! > Could you also post iTunes link here? > > thanks, > Slav > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Thu Mar 31 19:40:09 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 31 Mar 2011 17:40:09 +0000 (GMT) Subject: [erlang-questions] Re: List comprehension questions In-Reply-To: <OF73F2F9C4.4B183BD4-ONC1257863.003747FA-C1257863.003F28EF@notes.edfgdf.fr> Message-ID: <818442741.49161301593209500.JavaMail.root@zimbra> List comprehensions work as a combined lists:map and lists:filter. So you map over a list filtering out elements you don't want in the output list. They have the added functionality that they can handle this for nested lists. The expanded code is in fact quite simple and is close to what you probably write yourself. So order is preserved and their is no call to lists:reverse when generating the output list. Robert ----- "Olivier BOUDEVILLE" <olivier.boudeville@REDACTED> wrote: > > Hi, > > Reading http://www.erlang.org/doc/reference_manual/expressions.html#id77052 it is not obvious that the order induced by a single generator is necessarily preserved by the comprehension, as nevertheless suggested by all experiments like: > > 1> [ X || X <- [1,2,3] ]. > [1,2,3] > > I suppose this order preservation is true. Then, I imagine that, internally, list comprehensions are translated to code very similar to the one that would be produced if the developer had written an appropriate accumulator-based tail-recursive function? > > If so, does it imply that using a list comprehension on a semantically unordered set of elements will involve a useless reversal (like lists:reverse/1) of the resulting list? > > Thanks in advance for any answer, > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. > > E-mail communication cannot be guaranteed to be timely secure, error or virus-free. > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/7ac25807/attachment.htm> From bob@REDACTED Thu Mar 31 19:44:42 2011 From: bob@REDACTED (Bob Cowdery) Date: Thu, 31 Mar 2011 18:44:42 +0100 Subject: [erlang-questions] Re: linked-in driver In-Reply-To: <AANLkTinqYpLvWt_hKGjf_uj7c1vkadBJeYhtwmJU7DqX@mail.gmail.com> References: <4D9366BF.6080106@bobcowdery.plus.com> <AANLkTinqYpLvWt_hKGjf_uj7c1vkadBJeYhtwmJU7DqX@mail.gmail.com> Message-ID: <4D94BD8A.5080105@bobcowdery.plus.com> I did, soon after I sent the message. It's the same trap I fell into last time as I remember. All working now. On 31/03/2011 10:13, Tim Watson wrote: >> When I run it I get arity of 0 when it should be 2. >> >> What am I doing wrong? > Did you remember to ei_decode_version first? From ulf.wiger@REDACTED Thu Mar 31 20:04:39 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 31 Mar 2011 20:04:39 +0200 Subject: [erlang-questions] file operations on zipped files (Re: how to add a module to preLoaded?) In-Reply-To: <BEC404E6-42CE-4C6F-BC0E-00A93D0E8AFE@erlang-solutions.com> References: <BEC404E6-42CE-4C6F-BC0E-00A93D0E8AFE@erlang-solutions.com> Message-ID: <4BEEE64D-1FA3-41B9-B97B-1AFB0304BD9E@erlang-solutions.com> So after finding and trying 'make install', I finally noticed that my new module was in fact loaded, even though the {preLoaded, ?} entry in the boot script didn't say so. As a result, I still don't know which step actually made the difference, nor how to make the boot script reflect the change. Anyway, this now works: This patch makes most file operations transparent to whether the file is in a normal file system or in a zip archive. git fetch git@REDACTED:uwiger/otp.git uw-prim_file-reads-zip $ $ERL_TOP/bin/erl -pa run_proper.ez/proper/ebinErlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> compile:file("test/run_proper_test.erl",[debug_info]). {ok,run_proper_test} 2> beam_lib:chunks("run_proper_test.beam",[abstract_code]). {ok,{run_proper_test, [{abstract_code, {raw_abstract_v1, [{attribute,1,file,{"test/run_proper_test.erl",1}}, {attribute,1,module,run_proper_test}, {attribute,0,export,[]}, {attribute,2,compile,export_all}, {attribute,1,file, {"run_proper.ez/proper/include/proper.hrl",1}}, <====== HERE {attribute,1,file, {"run_proper.ez/proper/include/proper_common.hrl",1}}, <=== HERE {attribute,24,file, {"run_proper.ez/proper/include/proper.hrl",24}}, <=== HERE Note that the compiler fetched proper.hrl et al from the run_proper.ez zip file, which contains PropEr as a single zip file. The idea (see http://github.com/esl/run_eqc) was that it should be possible to deliver an application as a single archive, supporting command-line operations, interactive code loading, and compilation including hrl files, parse_transforms etc., without having to unpack the archive. All (?) sensible file operations on a read-only filesystem now work on files within a zip archive, as far as I can tell. The change in prim_file gives no overhead in the normal case, except for an extra pattern-match on the error value, and - if {error, enotdir}, a scan of the file name to see if there is a .ez file in the path. If so, the corresponding action is mapped on the archive. If the file is to be opened, as with epp, a "read_file()" is done, and the contents moved into a ram_file, which allows the same operations as prim_file. For this last case, I needed prim_ram_file in preloaded? A minor quirk is that code:lib_dir/1 has been defined to return a path *outside* the zip archive, presumably because you cannot load DLLs from an archive. Epp uses code:lib_dir/1 too, so even with the above changes, epp would fail to find the .hrl files. I solved this by having epp call code:where_is_file(Lib ++ ".app") and then finding the include from there. If that fails, it calls code:lib_dir/1 as before. I've not written a proper commit message (or actually sent this to erlang-patches), nor have I updated documentation or written test suites. If this approach is rejected on principle, I'd rather not waste the time, so, feedback first, please. BR, Ulf W On 31 Mar 2011, at 18:20, Ulf Wiger wrote: > > I'm trying to add a preloaded module for a patch I'm working on, but this turns out to be exceptionally difficult. > > I found the ./otp_build update_preloaded, which works great for existing preloaded files. > > Now trying to add (in my case) prim_ram_file, to the list of preloaded modules, I've so far found the list hard-coded in the following places: > > - erts/preloaded/src/Makefile > - $ERL_TOP/Makefile.in > - lib/sasl/src/systools_make.erl > > I feel that I have updated all the hard-coded representations, and tried various combinations of > - ./otp_build update_preloaded > - make > - make release > - ./otp_build setup > - ./otp_build boot > - ./otp_build update_primary > > None of them seem to actually change the start scripts - not that I'm even sure that this would make a difference, since the whole preloading is a deep dark mystery. > > What is the magic incantation that I'm missing? > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Thu Mar 31 20:34:12 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 31 Mar 2011 19:34:12 +0100 Subject: [erlang-questions] Re: linked-in driver In-Reply-To: <4D94BD8A.5080105@bobcowdery.plus.com> References: <4D9366BF.6080106@bobcowdery.plus.com> <AANLkTinqYpLvWt_hKGjf_uj7c1vkadBJeYhtwmJU7DqX@mail.gmail.com> <4D94BD8A.5080105@bobcowdery.plus.com> Message-ID: <AANLkTikki06r2DvV9RTTi9cHfTR788qAMVZfpwvu_aV4@mail.gmail.com> On 31 March 2011 18:44, Bob Cowdery <bob@REDACTED> wrote: > I did, soon after I sent the message. It's the same trap I fell into > last time as I remember. All working now. > Yep, I've forgotten that a fair few times too. :) From gleber.p@REDACTED Thu Mar 31 20:40:31 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 31 Mar 2011 20:40:31 +0200 Subject: [erlang-questions] Configuring sasl to write rb-compatible logs Message-ID: <AANLkTimk8+0RJ18KwXD=ZDndEx3Ja+t6GbDp7iaVVsn6@mail.gmail.com> Hi I'm digging into using rb for log viewing. As far as I understood from documentation, some blogs and other emails here it is enough to define the following parameters in the sasl application's config: error_logger_mf_dir error_logger_mf_maxbytes error_logger_mf_max_files Here's an output of my test: ============================================ gleber@REDACTED:~/tmp/test$ ls rb.config test.sh gleber@REDACTED:~/tmp/test$ cat rb.config [ {sasl, [ {error_logger_mf_dir, "."}, {error_logger_mf_maxbytes, 1024}, {error_logger_mf_max_files, 10} ] } ]. gleber@REDACTED:~/tmp/test$ cat test.sh #!/bin/sh exec erl -sname test \ -config rb \ -boot start_sasl gleber@REDACTED:~/tmp/test$ ./test.sh Erlang R14A (erts-5.8) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] =PROGRESS REPORT==== 31-Mar-2011::20:36:09 === supervisor: {local,sasl_safe_sup} started: [{pid,<0.41.0>}, {name,alarm_handler}, {mfargs,{alarm_handler,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 31-Mar-2011::20:36:09 === supervisor: {local,sasl_safe_sup} started: [{pid,<0.42.0>}, {name,overload}, {mfargs,{overload,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 31-Mar-2011::20:36:09 === supervisor: {local,sasl_sup} started: [{pid,<0.40.0>}, {name,sasl_safe_sup}, {mfargs, {supervisor,start_link, [{local,sasl_safe_sup},sasl,safe]}}, {restart_type,permanent}, {shutdown,infinity}, {child_type,supervisor}] =PROGRESS REPORT==== 31-Mar-2011::20:36:09 === supervisor: {local,sasl_sup} started: [{pid,<0.43.0>}, {name,release_handler}, {mfargs,{release_handler,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =PROGRESS REPORT==== 31-Mar-2011::20:36:09 === application: sasl started_at: test@REDACTED Eshell V5.8 (abort with ^G) (test@REDACTED)1> rb:start(). {error,{"cannot read the index file", {child,undefined,rb_server, {rb,start_link,[[]]}, temporary,brutal_kill,worker, [rb]}}} (test@REDACTED)2> =CRASH REPORT==== 31-Mar-2011::20:36:11 === crasher: initial call: rb:init/1 pid: <0.49.0> registered_name: [] exception exit: "cannot read the index file" in function gen_server:init_it/6 ancestors: [sasl_sup,<0.38.0>] messages: [] links: [<0.39.0>] dictionary: [] trap_exit: true status: running heap_size: 233 stack_size: 24 reductions: 266 neighbours: (test@REDACTED)2> q(). ok (test@REDACTED)3> gleber@REDACTED:~/tmp/test$ ls rb.config test.sh ============================================ Am I missing something in the configuration of sasl or is there anything else I have to do to make sasl attach error_logger_mf_h to error logger? Best, Gleb Peregud From ulf.wiger@REDACTED Thu Mar 31 21:15:16 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 31 Mar 2011 21:15:16 +0200 Subject: [erlang-questions] Re: Distributed application takeover In-Reply-To: <AANLkTikGaU2B5B4Udk7gyJGgekMPEGPD5DdOkLEajoRO@mail.gmail.com> References: <AANLkTi=RuSSaeR_w4TLFJeKW6-0DhfQ5a78RCZ1Uq668@mail.gmail.com> <0DBAD7AB-FE6B-4A54-A07B-C8ABAB6E8ACF@erlang-solutions.com> <AANLkTikGaU2B5B4Udk7gyJGgekMPEGPD5DdOkLEajoRO@mail.gmail.com> Message-ID: <61A1080E-B2E8-4CE1-9A47-2401761169BA@erlang-solutions.com> Hi Jeroen, What we used to do was to use start phases and global:re_register_name/2 Application start phases can be used to do e.g. global name registration in an ordered fashion. We had, for historical reasons, the following start phases at AXD 301: - start: processes started, basic initialisation - takeover: ignored unless application start type was {takeover, FromNode} - go: to register global names, do other forms of complex initialisation, etc. In the takeover phase, the takeover callback would typically do gen_server calls to the application processes. These could in their turn do a gen_server call to their counterpart on the other side, fetching state, telling the other side to start forwarding requests, etc. A good order of doing things would be to first re_register the global name, then call the other process to fetch state. See: http://www.erlang.org/doc/man/app.html http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3 http://www.erlang.org/doc/man/global.html#re_register_name-2 BR, Ulf W On 31 Mar 2011, at 14:36, Jeroen Koops wrote: > Hi Ulf, > > Thanks for the explanation, the message-flow is now pretty clear. > > Actually, the reason I was looking into it was to see if I could come up with a patch that makes it possible to configure an alternative takeover behaviour, since I find it inconvenient that during a takeover the application is running on two nodes at the same time for a while, which leads to clashes with globally registered names, and problems with processes that assume they have exclusive access to data. It would be nice to be able to configure takeover behaviour in such a way that the application is only started on the node that is taking over after it has been stopped on the original node. > > However, I've come to the conclusion that the whole application distribution is way too complex to start poking around, so instead I now use a mechanism where, whenever an application is started with { takeover, OtherNode } as StartType, it simply terminates all the children of the application's top-level supervisor on OtherNode before proceeding with the startup. This is a bit clunky, and a feel a bit sorry for the application on OtherNode being left behind as an empty shell, with all its children killed, waiting for the application controller to put it out of its misery, but it gets the job done... > > > On Wed, Mar 30, 2011 at 4:00 PM, Ulf Wiger <ulf.wiger@REDACTED> wrote: > > Hi Jeroen, > > This is a bit tricky, and can be very confusing for someone trying to build their own cluster controller. :) > > The stopping of the application is done by the local application_controller when it is told that the application is running somewhere else. In other words, dist_ac kindly forwards the information to the local AC, and it is the local AC that in this particular case takes responsibility for 'knowing' that the local instance should be stopped. > > See application_controller:handle_application_started/3 > > https://github.com/erlang/otp/blob/OTP_R14B02/lib/kernel/src/application_controller.erl#L913 > > (The dirty deed actually happens on line 946). > > The reason I know this, is that in the AXD 301, we ran multiple instances of some applications, distributed across several mated pairs - each instance having its own standby node. This can be done by writing an own distributed AC, but it has to be smart enough to know _when_ to forward the {started, Node} status to the local AC; if an instance was running locally - and _was supposed to_ do so (i.e. not involved in a takeover), the distributed AC had to suppress this message. > > BR, > Ulf W > > On 30 Mar 2011, at 14:11, Jeroen Koops wrote: > >> Hi, >> >> I'm trying to find out how the distributed application controller works internally. I'm especially interested in the implementation of an application takeover. >> >> In case an application runs on node A, and is taken over by node B, what should happen is that it is first started on node B, so that there are two instances of the application running simultaneously for a brief period of time, and then stopped on node A. >> >> However, I cannot figure out where this stopping happens in dist_ac.erl. If I understand correctly, this should happen in response to a ac_application_run message from the application_controller. This message is received by the dist_ac on node B, and a dist_ac_app_started message is then broadcast to the dist_acs on all connected nodes. The dist_ac of node A receives this message, notices that the application is still running locally, and decides to shut down the application on its own node -- at least that is what the comments say (dist_ac.erl, line 529): >> >> %% Another node tookover from me; stop my application >> %% and update the running list. >> >> But all I can see is that the dist_ac's list of applications is updated to indicate that the application is no longer running locally -- I cannot find where the application_controller is instructed to actually shutdown the application. >> >> Can anyone point me in the right direction? >> >> Thanks, >> >> Jeroen _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/7b9d4527/attachment.htm> From mjtruog@REDACTED Thu Mar 31 22:04:59 2011 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 31 Mar 2011 13:04:59 -0700 Subject: [erlang-questions] Erlang External Term Format for PIDs Message-ID: <4D94DE6B.2050604@gmail.com> Hi, I was wondering why the Erlang pid includes both the Node, which is variable length, and the ID integer. Isn't the ID integer node specific such that every node has a unique index? I am wondering for unique id generation (not the make ref type, but rather UUID code here https://github.com/okeuday/uuid). The ID field is explained here http://www.erlang.org/doc/apps/erts/erl_ext_dist.html#REFERENCE_EXT , since it is the same integer used in references. I am assuming that the external term format wants to be specific to a node as it is currently named, in case it changes names in the future (not sure if that works, but I assume it is possible somehow). Is that the reason both the Node name and the ID exist together? The main reason I ask, is just to make sure the ID integer is truly unique for an Erlang node's view of the connected nodes when setting pid data without needing to create a hash from the node name if something needs to be pid-specific without being unique among all connected Erlang nodes (the UUID RFC prefers the MAC address, which may be a worse choice than a hash of the node name, especially since it depends on undocumented function calls). Thanks, Michael From stanhope@REDACTED Thu Mar 31 23:40:33 2011 From: stanhope@REDACTED (Phil Stanhope) Date: Thu, 31 Mar 2011 17:40:33 -0400 Subject: [erlang-questions] Re: file operations on zipped files (Re: how to add a module to preLoaded?) In-Reply-To: <4BEEE64D-1FA3-41B9-B97B-1AFB0304BD9E@erlang-solutions.com> References: <BEC404E6-42CE-4C6F-BC0E-00A93D0E8AFE@erlang-solutions.com> <4BEEE64D-1FA3-41B9-B97B-1AFB0304BD9E@erlang-solutions.com> Message-ID: <AANLkTinNYNSLtE0fHUX-+Z0ELUCnXUwX=yNVp25BVX9=@mail.gmail.com> Very cool. I can see how this will make deployments very easy. On Mar 31, 2011 2:05 PM, "Ulf Wiger" <ulf.wiger@REDACTED> wrote: So after finding and trying 'make install', I finally noticed that my new module was in fact loaded, even though the {preLoaded, ?} entry in the boot script didn't say so. As a result, I still don't know which step actually made the difference, nor how to make the boot script reflect the change. Anyway, this now works: This patch makes most file operations transparent to whether the file is in a normal file system or in a zip archive. git fetch git@REDACTED:uwiger/otp.git uw-prim_file-reads-zip $ $ERL_TOP/bin/erl -pa run_proper.ez/proper/ebinErlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> compile:file("test/run_proper_test.erl",[debug_info]). {ok,run_proper_test} 2> beam_lib:chunks("run_proper_test.beam",[abstract_code]). {ok,{run_proper_test, [{abstract_code, {raw_abstract_v1, [{attribute,1,file,{"test/run_proper_test.erl",1}}, {attribute,1,module,run_proper_test}, {attribute,0,export,[]}, {attribute,2,compile,export_all}, {attribute,1,file, {"run_proper.ez/proper/include/proper.hrl",1}}, <====== HERE {attribute,1,file, {"run_proper.ez/proper/include/proper_common.hrl",1}}, <=== HERE {attribute,24,file, {"run_proper.ez/proper/include/proper.hrl",24}}, <=== HERE Note that the compiler fetched proper.hrl et al from the run_proper.ez zip file, which contains PropEr as a single zip file. The idea (see http://github.com/esl/run_eqc) was that it should be possible to deliver an application as a single archive, supporting command-line operations, interactive code loading, and compilation including hrl files, parse_transforms etc., without having to unpack the archive. All (?) sensible file operations on a read-only filesystem now work on files within a zip archive, as far as I can tell. The change in prim_file gives no overhead in the normal case, except for an extra pattern-match on the error value, and - if {error, enotdir}, a scan of the file name to see if there is a .ez file in the path. If so, the corresponding action is mapped on the archive. If the file is to be opened, as with epp, a "read_file()" is done, and the contents moved into a ram_file, which allows the same operations as prim_file. For this last case, I needed prim_ram_file in preloaded? A minor quirk is that code:lib_dir/1 has been defined to return a path *outside* the zip archive, presumably because you cannot load DLLs from an archive. Epp uses code:lib_dir/1 too, so even with the above changes, epp would fail to find the .hrl files. I solved this by having epp call code:where_is_file(Lib ++ ".app") and then finding the include from there. If that fails, it calls code:lib_dir/1 as before. I've not written a proper commit message (or actually sent this to erlang-patches), nor have I updated documentation or written test suites. If this approach is rejected on principle, I'd rather not waste the time, so, feedback first, please. BR, Ulf W On 31 Mar 2011, at 18:20, Ulf Wiger wrote: > > I'm trying to add a preloaded module for a patch I'm working on, but this turns out to be exceptionally difficult. > > I found the ./otp_build update_preloaded, which works great for existing preloaded files. > > Now trying to add (in my case) prim_ram_file, to the list of preloaded modules, I've so far found the list hard-coded in the following places: > > - erts/preloaded/src/Makefile > - $ERL_TOP/Makefile.in > - lib/sasl/src/systools_make.erl > > I feel that I have updated all the hard-coded representations, and tried various combinations of > - ./otp_build update_preloaded > - make > - make release > - ./otp_build setup > - ./otp_build boot > - ./otp_build update_primary > > None of them seem to actually change the start scripts - not that I'm even sure that this would make a difference, since the whole preloading is a deep dark mystery. > > What is the magic incantation that I'm missing? > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/b7bb3a67/attachment.htm>